FTP hangs with urllib.urlretrieve

[Oleg Broytmann]

On Tue, 7 Mar 2000, Michal Wallace (sabren) wrote:
> >    I am developing/running URL checker (sorry, yet another one :). And
> > found that I have consistent hangings on some FTP URLs.
> > 
> This is not limited to ftp urls... urllib just hangs sometimes.

   Yes, but I've never see an HTTP URL that my URL checker will hangs
always. And there are some FTP URLs on which the robot always hangs
forever. :(

> I have a bot that uses alarm(), plus some code I found on
> comp.lang.python's archives.. but it still hangs on occasion. I have

   That's because alarm() uses signal, and Python is poor on handling
signals :(

> no clue how to fix it.. And wound up just writing a second cron job
> that would come by and kill hung versions. :/

   There are 2.5 ways to solve this:

1. Non-blocking sockets; asyncore

2. Multitasking:
   a) fork
   b) multithreading.

Oleg.
---- 
    Oleg Broytmann      Foundation for Effective Policies      phd at phd.russ.ru
           Programmers don't die, they just GOSUB without RETURN.