How many connections can accept a 'binded' socket?

[Alex Martelli]

Laurent Pointal <laurent.pointal at limsi.fr> wrote:

> billiejoex a écrit :
> > Hi,
> > I'm writing a small asyncore-based server application serving a lot of
> > clients. When I have to handle more than 1021 client simoultaneously
> > the 'binded' socket object raises an error:
> > 
> > [...]
> > connections: 1018
> > connections: 1019
> > connections: 1020
> > connections: 1021
> > Traceback (most recent call last):
> >   File "asyncore_client.py", line 31, in <module>
> >   File "asyncore.py", line 191, in loop
> >   File "asyncore.py", line 138, in poll
> >   File "asyncore.py", line 80, in write
> >   File "asyncore.py", line 76, in write
> >   File "asyncore.py", line 395, in handle_write_event
> >   File "asyncore_client.py", line 24, in handle_connect
> >   File "asyncore_client.py", line 9, in __init__
> >   File "asyncore.py", line 257, in create_socket
> >   File "socket.py", line 156, in __init__
> > socket.error: (24, 'Too many open files')
> > 
> > I just wanna know: is there a way to know how many connections can
> > accept a 'binded' socket BEFORE getting such error? Maybe
> > socket.SOMAXCONN could help me?
> 
> Here you get out of file descriptors, I dont think SOMAXCONN would help.
> 
> Under Linux (maybe Unix), there is ulimit -n nnn to setup the maximum
> number of files descriptors. I don't know its upper limit (maybe a
> kernel compile time information).

A shell command

ulimit -Hn

should report on the hard-limit of the number of open file descriptors;
just ulimit -n should report on the current soft-limit.

If you're going to pass the fd's to select, as asyncore does by default,
a separate limit of 1024 is unfortunately likely to apply anyway; so,
once that ulimit is raised, you may want to pass argument use_poll as
true to asyncore.loop.  The performance of poll with a huge number of
sockets may however not be all that shiny.  Better mechanisms, such as
epoll or kqueue, I believe, are not available for asyncore, even if your
OS supports them; to serve thousands of open sockets with good
performance, you may need to switch to Twisted (which offers many more
implementations of the abstract Reactor interface -- you don't _have_ to
use Twisted's higher layers if you don't want to).


Alex