Scrapy And Django Import Error

When I am calling Spider through a Python script, it is giving me an ImportError: ImportError: No module named app.models My items.py is like this: from scrapy.item import Item, F

Solution 1:

Perhaps your problem is that you are importing the spider before the settings. The ImportError might come from the from app.models import Person in your items.py.

So, import your spider after you set up the settings:

crawler.configure()

from final.aqaq.aqaq.spiders.spider importaqaqspiderspider= aqaqspider(domain='aqaq.com')

crawler.crawl(spider)

Solution 2:

I writed this post in the Medium a time ago, perhaps it can help you!

https://medium.com/@tiago.piovesan.tp/make-a-crawler-with-django-and-scrapy-a41adfdd24d9

This is the integrate config between the libs: crawler/settings.py

import os
import sys
sys.path.append(os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), ".."))
os.environ['DJANGO_SETTINGS_MODULE'] = 'myweb.settings'
import django
django.setup()