infinite loop with httplib?

Karl Witmann calnerd at gmail.com
Fri Jul 9 03:18:37 EDT 2004


Anyone have experience with httplib getting stuck reading the first line
of a file?

Most urls work fine (of course), but I tried the following:
  http://216.117.129.109/robots.txt

Both Firefox and IE return:
   fopen: No such file or directory
   httpd: could not open document config file /etc/httpd/conf/httpd.conf
A misconfigured server...

But for my test script:
>>> import urllib2
>>> f = urllib2.urlopen('http://216.117.129.109')

>>> print f.readline()
fopen: No such file or directory
>>> print f.readline()
fopen: No such file or directory
>>> print f.readline()
fopen: No such file or directory

>>> print f.read()
Traceback (most recent call last):
  File "<interactive input>", line 1, in ?
  File "C:\Python23\lib\httplib.py", line 1156, in read
    assert not self._line_consumed and self._line_left
AssertionError

Worse, the robotparser uses this code and thus ALL available memory:
>>> import robotparser
>>> rp = robotparser.RobotFileParser()
>>> rp.set_url('http://216.117.129.109')
>>> rp.read()
a few minutes later, the kernel kills the process....

Any ideas?

I'd be fine if this returned an exception rather than a kernel panic.



More information about the Python-list mailing list