Urllib vs. FireFox

[Mike Driscoll]

On Oct 24, 2:53 pm, Rex <rex.eastbou... at gmail.com> wrote:
> Right. If you want to get the same results with your Python script
> that you did with Firefox, you can modify the browser headers in your
> code.
>
> Here's an example with urllib2:http://vsbabu.org/mt/archives/2003/05/27/urllib2_setting_http_headers...
>
> By the way, if you're doing non-trivial web scraping, the mechanize
> module might make your work much easier. You can install it with
> easy_install.http://wwwsearch.sourceforge.net/mechanize/

Or if you just need to query stuff on Amazon, then you might find this
module helpful:

http://pypi.python.org/pypi/Python-Amazon/

-------------------
Mike Driscoll

Blog:   http://blog.pythonlibrary.org
Python Extension Building Network:     http://www.pythonlibrary.org