How To Build Scrapy Environment

Peter Otten __peter__ at web.de
Mon Sep 23 04:37:22 EDT 2013


YetToCome wrote:

> I have already install Twisted, zope.interface, w3lib, libxml2, etc,but it
> still can not be built,here is the error message:
> 
> Traceback (most recent call last):
>   File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
>     "__main__", fname, loader, pkg_name)
>   File "C:\Python27\lib\runpy.py", line 72, in _run_code
>     exec code in run_globals
>   File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 167, in
>   <module>
>     execute()
>   File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 142, in
>   execute
>     _run_print_help(parser, _run_command, cmd, args, opts)
>   File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 88, in
>   _run_print
> _help
>     func(*a, **kw)
>   File "C:\Python27\lib\site-packages\scrapy\cmdline.py", line 149, in
>   _run_comm
> and
>     cmd.run(args, opts)
>   File "C:\Python27\lib\site-packages\scrapy\commands\crawl.py", line 47,
>   in run
>     crawler = self.crawler_process.create_crawler()
>   File "C:\Python27\lib\site-packages\scrapy\crawler.py", line 142, in
>   create_cr
> awler
>     self.crawlers[name] = Crawler(self.settings)
>   File "C:\Python27\lib\site-packages\scrapy\crawler.py", line 23, in
>   __init__
>     self.spiders = spman_cls.from_crawler(self)
>   File "C:\Python27\lib\site-packages\scrapy\spidermanager.py", line 35,
>   in from
> _crawler
>     sm = cls.from_settings(crawler.settings)
>   File "C:\Python27\lib\site-packages\scrapy\spidermanager.py", line 31,
>   in from
> _settings
>     return cls(settings.getlist('SPIDER_MODULES'))
>   File "C:\Python27\lib\site-packages\scrapy\spidermanager.py", line 22,
>   in __in
> it__
>     for module in walk_modules(name):
>   File "C:\Python27\lib\site-packages\scrapy\utils\misc.py", line 66, in
>   walk_mo
> dules
>     submod = __import__(fullpath, {}, {}, [''])
>   File "blog_crawl\spiders\dmoz_spider.py", line 1, in <module>
>     class DmozSpider(BaseSpider):
> NameError: name 'BaseSpider' is not defined
> 
> someone tells me why, pls

Assuming you are working your way through the tutorial at

http://doc.scrapy.org/en/latest/intro/tutorial.html#our-first-spider

you probably forgot to import the BaseSpider class with

from scrapy.spider import BaseSpider

as shown in the code snippet in the "Our first Spider" section of the 
tutorial.





More information about the Python-list mailing list