socket.sendall(), non-blocking sockets, and multi-threaded socket sending

[Alan Kennedy]

[Tim Black]
> My application requires sending a large piece (~2MB) of data to
> several devices on a network via TCP sockets. I have experimented with
> different methods for doing this and this has raised some questions
> about the implementation of Python sockets.
> 
> (both methods use blocking sockets)
> 
> Method 1: Calls socket.sendall(data) for each device in sequence, all
> in a single thread.
> 
> Method 2: Each socket has its own thread that calls
> socket.send(datachunk) iteratively until all data is sent.
> 
> So Method 1 sends all data to device1, then send all data to device2,
> etc... until all data is sent to all devices. Method 2 sends chunks of
> data to all devices in parallel. What I am seeing is that Method 1 is
> faster than Method 2. Given that the bottleneck is the actual sending
> of data on the sockets and not some latency in processing the message
> at the other end, this result makes sense to me. Method 2 involves
> sending the data in smaller chunks, whereas with Method 1 the
> chunksize is selected by the TCP/IP stack implementation, which is
> probably more efficient. Also, Method 2 entails context switching
> between the send threads.
> 
> It turns out that using Method 1, all sendall() calls return
> immediately. I am assuming at this point that my app is in the hands
> of the TCP/IP stack implementation on my machine (win32-XP). Can
> anyone explain what happens inside sendall() for Win32?
> 
> Also, I would like to get advice and opinions on what is the most
> efficient way to broadcast data to several devices on a network.

I'd love to have enough time to get into discussing your findings 
about sockets. But unfortunately, I don't :-(

But I did want to point out a module that you may not have come across 
that makes the job of distributing information peer-to-peer over LANs 
very easy: the spread module. I highly recommend that you take a look 
over it: it could save you a lot of wheel-reinvention.

http://www.python.org/other/spread/

HTH,

-- 
alan kennedy
------------------------------------------------------
check http headers here: http://xhaus.com/headers
email alan:              http://xhaus.com/contact/alan