simple httplib and urllib timeout question

[darrell at rubina.funkomite.com]

"Michal Wallace (sabren)" <sabren at manifestation.com> wrote:
> On Tue, 11 Apr 2000 darrell at rubina.funkomite.com wrote:

>> Just a quick question, im writing a small webbot for a python
>> learning experience and it comes across some sites that will make it
>> hang for a while, if the site is down or extremely slow etc, is
>> there a way to set a faster timeout for httplib . Or is there a
>> better way to handle it ?

> Yes. The better way is to use asyncore whenever httplib/urllib uses a
> normal socket. Unfortunately, this requires breaking urllib into
> smaller pieces, and nobody seems to have done it yet. I'm actually
> working on the exact same problem (and posting regular updates
> at http://www.linkwatcher.com/ ) 

> I sent an email to Jeremy Hylton, who's working on urllib2. He's
> swamped, but willing to accept a patch that would add this feature
> to urllib2.. I think I'll wind up taking a crack at it tonight.

> There's also a partial solution using a different method if you look
> at...  http://www.linkwatcher.com/src/blogbot/timeout.py

> there's an example script called blogbot.py there, too.. it uses
> timeout.py and another hack with signal.alarm(), but neither really
> works all that well..

> hope that helps.

> meanwhile... HEY OLEG! -> have you done anything w/ this?

> Cheers,

> - Michal
> -------------------------------------------------------------------------
> http://www.manifestation.com/         http://www.linkwatcher.com/metalog/
> -------------------------------------------------------------------------

Thanks Michal , this helps alot :>