[Patches] [ python-Patches-499513 ] robotparser.py fails on some URLs

[noreply@sourceforge.net]

Patches item #499513, was opened at 2002-01-04 10:21
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=499513&group_id=5470

Category: Library (Lib)
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Bastian Kleineidam (calvin)
Assigned to: Nobody/Anonymous (nobody)
Summary: robotparser.py fails on some URLs

Initial Comment:
I am using Python 2.1.1.

The URL http://www.chaosreigns.com/robots.txt results
in an empty RobotParser object.
Reason is that the file object returned from the
URLOpener does not have a readlines() attribute.

I patched the robotparser.py to use readline() instead
of readlines().
Furthermore I removed the unnecessary redirection limit
code which is already in FancyURLopener.


Greetings, Bastian


----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=499513&group_id=5470