Non blocking sockets with select.poll() ?

[Jean-Paul Calderone]

On Fri, 4 May 2007 15:05:46 +0300, Maxim Veksler <hq4ever at gmail.com> wrote:
>On 5/4/07, Maxim Veksler <hq4ever at gmail.com> wrote:
>>On 5/4/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> > On Fri, 4 May 2007 13:04:41 +0300, Maxim Veksler <hq4ever at gmail.com> 
>>wrote:
>> > >Hi,
>> > >
>> > >I'm trying to write a non blocking socket port listener based on
>> > >poll() because select is limited to 1024 fd.
>> > >
>> > >Here is the code, it never gets to "I did not block" until I do a
>> > >telnet connection to port 10000.
>> > >
>> >
>> > What were you expecting?
>> >
>>
>>I'll try to explain.
>>I'm doing a little experiment: Capturing the whole tcp 1-65535 range
>>of the machine, allowing me to connect to the my service on the
>>machine on every port. I know that it's probably the most dumb thing
>>to do with TCP/IP communication please don't forget it's an
>>experiment.
>
>[snip]
>
>I think I got it working now :)
>
>"""
>#!/usr/bin/env python
>import socket
>import select
>
>class PollingSocket(socket.socket):
>
>
>    def __init__(self, port_number):
>        self.__poll = select.poll()
>        self.tcp_port_number = port_number
>
>        socket.socket.__init__(self, socket.AF_INET, socket.SOCK_STREAM)
>        self.setblocking(0)
>        self.bind(('0.0.0.0', self.tcp_port_number))
>        self.listen(5)
>        self.__poll.register(self)
>
>    def poll(self, timeout = 0):
>        return self.__poll.poll(timeout)
>
>def debugPollingSocket(port_num):
>    print "BIND TO PORT: ", port_num
>    return PollingSocket(port_num)
>
>all_sockets = map(debugPollingSocket, xrange(10000, 19169))
>
>print "We have this in stock:"
>for nb_active_socket in all_sockets:
>    print nb_active_socket.tcp_port_number
>
>while 1:
>    for nb_active_socket in all_sockets:
>        print "Asking", nb_active_socket.tcp_port_number
>        if nb_active_socket.poll(0):
>            print "Found", nb_active_socket.tcp_port_number
>            conn, addr = nb_active_socket.accept()
>            while 1:
>                data = conn.recv(1024)
>                if not data: break
>                conn.send(data)
>            conn.close()
>"""
>

This will only handle one connection at a time, of course.  The polling
it does is also somewhat inefficient.  Perhaps that's fine for your use
case.  If not, though, I'd suggest this version (untested):

  from twisted.internet import pollreactor
  pollreactor.install()

  from twisted.internet import reactor
  from twisted.protocols.wire import Echo
  from twisted.internet.protocol import ServerFactory

  f = ServerFactory()
  f.protocol = Echo
  for i in range(10000, 19169):
      reactor.listenTCP(i, f)
  reactor.run()

This will handle traffic from an arbitrary number of clients at the same
time and do so more efficiently than the loop in your version.  You can
also try epollreactor instead of pollreactor, if the version of Linux you
are using supports epoll, for even better performance.

Jean-Paul