Slow Queue.queue? (was: slow network)

Laszlo Nagy gandalf at shopzeus.com
Thu Jan 15 04:22:06 EST 2009


>
> then the speed goes up to 64 messages/sec on windows and 500 
> messages/sec on Linux.
Finally I could reach 1500 messages/sec without using the queue. If I 
comment out one line (use the queue instead of direct write into socket) 
then speed decreases to 40-60 messages/sec. I don't understand why the 
slow version is slower by a factor of 40?

Fast version:

    def send_message(self,sender,recipient,msgtype,body,timeout=3600):
        
self.write_str(self.serializer.serialize([sender,recipient,msgtype,body]))

Slow version:

    def send_message(self,sender,recipient,msgtype,body,timeout=3600):
        self.outgoing.put(self.serializer.serialize([
            sender,recipient,msgtype,body
        ]),1,timeout)

plus this method, executed in a different thread:

    def _process_outgoing(self):
        try:
            while not self.stop_requested.isSet():
                data_ok = False
                while not self.stop_requested.isSet():
                    try:
                        data = self.outgoing.get(1,1)
                        data_ok = True
                        break
                    except Queue.Empty:
                        pass
                if data_ok:
                    self.write_str(data)
        except Exception, e:
            if self.router:
                if not isinstance(e,TransportClosedError):
                    self.router.logger.error(dumpexc(e))
                self.router.unregister_endpoint(self)
            self.shutdown()
            raise SystemExit(0)




More information about the Python-list mailing list