How To Execute Scrapy Shell "URL" With Notebook
i am trying to scrapy and there is scrapy shell 'URL' command, executing console ipython with a response object from the URL. but i want to do that thing with notebook. is there an
Solution 1:
Stumbled across this unanswered question looking for the same answer. The closest thing I can come up right now is this:
import requests
from scrapy.http import TextResponse
r = requests.get('http://stackoverflow.com/')
response = TextResponse(r.url, body=r.text, encoding='utf-8')
Then you can debug response using usual methods like xpath().
Post a Comment for "How To Execute Scrapy Shell "URL" With Notebook"