Sockets

[pehr anderson]

For this sort of spider app you might want to try timeoutsocket
which is an overlay on conventional python sockets that 
causes them to timeout after a specified number of seconds.

Being tied to an IP stack, your computer has a maximum of 65535 
active sockets. With the normal 2 hour timeout for IP sockets, 
your spider probably doesn't take long to fill up this entire space!
Check this by running netstat on linux. I don't know if windows
has a diagnostic tool for this...

	-pehr


David Arnold wrote:
> 
> -->"Aahz" == Aahz Maruch <aahz at netcom.com> writes:
> In article <8odh9e$qqf$1 at nnrp1.deja.com>, <karlsass at my-deja.com>
> wrote:
> 
>   >>  I have a set of spiders running on my system, The spiders
>   >> communicate with a urlserver & a processing server via internal
>   >> sockets. The spiders are the only ones using Inet sockets. Yet
>   >> after a while of say 1 or 2 spiders, a urlserver and process
>   >> server running , say for 3 hours, then all of a sudden the system
>   >> disallows any connections to the outside, eg inet sockets as well
>   >> as Netscape cannot connect to other web located resources.
> 
>   Aahz> Yeah, sounds like you're not properly closing down the sockets
>   Aahz> after you use them.
> 
> i had a bug yesterday, on WinNT (it was on someone else's system so
> i'm not sure about this), where using socket.close() does not close
> the underlying socket, byt socket.shutdown() does.
> 
> might be worth a go?
> 
> d