Scrapy - importing files from local, rather than www

nbw bwalker at itree.com.au
Mon May 7 03:57:22 EDT 2012


Hi everyone, I'm new to Python (loving it!) and Scrapy. I have a
question I just can't seem to get my head around. I can get a simple
Scrapy spider to pick up URLs and download them fine, but the HTML
files I have are stored locally. The reason for this, is for some
reason when I "Save As" the pages I get everything, whereas if Scrapy
runs over them it seems to miss certain areas where there's
Javascript.

So, I have them sitting in a directory (C:/scrapy_test) but can't for
the life of me get Scrapy to find them. Is there anyone who's had this
problem and solved it, or can help?

Any help is much appreciated.
Kind regards,
nbw



More information about the Python-list mailing list