how to handle network failures

[Jorgen Grahn]

On Fri, 2010-10-08, harryos wrote:
> hi
> I  am trying to write a DataGrabber which reads some data from given
> url..I made DataGrabber as a Thread and want to wait for some interval
> of time in case there is a network failure that prevents read().
> I am not very sure how to implement this
>
> class DataGrabber(threading.Thread):
>     def __init__(self,url):
>         threading.Thread.__init__(self)
>         self.url=url
>     def run(self):
>         data=self.get_page_data()
>         process_data(data)
>
>     def get_page_data():
>         try:
>             f=urllib.urlopen(self.url)
>             data=f.read(1024)
>         except IOError:
>             #wait for some time and try again
>             time.sleep(120)
>             data=self.get_page_data()
>         return data
>
> Is this the way to  implement the part where the thread waits and
> reads the  data again? Will this handle network failures?Can somebody
> please help?

You are using TCP sockets. When you get an error on one of those, the
TCP connection is dead (except for a few special cases like EAGAIN,
EINTR).

But you also risk *not* getting told and hanging forever, or anyway
for far longer than your application is likely to want to wait. For
example if the peer host is suddenly disconnected from the network --
TCP will keep trying, in case a connection suddenly reappears much
later.

Try provoking that situation and see what happens.

/Jorgen

-- 
  // Jorgen Grahn <grahn@  Oo  o.   .  .
\X/     snipabacken.se>   O  o   .