Slow Queue.queue? (was: slow network)

Laszlo Nagy gandalf at shopzeus.com
Thu Jan 15 03:33:43 EST 2009


I had this test program originally that sent message over TCP/IP. 
Messages where buffered two a Queue.Queue instances, one for incoming 
and one for outgoing.

#1. endpoint.send_message(msg) -> endpoint.outgoing.put(msg)
#2. endpoint._process_outgoing() is a thread, that does:  
endpoint.write_into_socket( endpoint.outgoing.get() )

On the other side:

#1 endpoint._process_incoming() is a thread, that does 
endpoint.incoming.put( endpoint.read_from_socket() )
#2 endpoing.recv_messages() -> return endpoint.incoming.get()

This could do 130 message pairs/sec on Linux (on 127.0.0.1) and only 20 
message pairs /sec on Windows XP.

If do not start the _process_outgoing() thread, and do not use the 
outgoing queue:

#1. endpoint.send_message(msg) -> endpoint.write_into_socket(msg)

then the speed goes up to 64 messages/sec on windows and 500 
messages/sec on Linux.

This has nothing to do with TCP/IP. The only change is that I'm not 
using a queue, and not using +1 thread. How can this slow down my 
program? Is it a problem with Queue.Queue, or with the +1 thread?

Thanks,

   Laszlo




More information about the Python-list mailing list