[Catalog-sig] threads and xmlrpc?

["Martin v. Löwis"]

> I'm running a threaded app using some calls via xmlrpc to pypi. What
> I'm trying to get is a to get a littler more responses in a shorter
> time, as I see that the bandwidth used by xmlrpc calls are minimal
> (<kb). The problem I run into is that connection is reset by peer
> after about 10min (~500 calls). I use a single connection and a queue
> of 8 threads to get the data. Would anybody have an example on how to
> run xmlrpc in a thread? Do I set multiple connections, or is there a
> setting to keep the connection live or reconnect if disconnected?

Using threads will not at all make it faster to communicate over a
single connection. For a single connection, all communication must
be serialized; you cannot issue a new request until the previous
request has completed. So you might as well just issue the requests
from a single thread.

> Also, please advice if you think that somehow I am overloading your
> servers. I've tasted some downloads speeds and I am sure you web
> browser can accept 100+ requests per second, but what about xmlrpc?
> Without threads I get <5 requests per second.

I think 5 requests per second is fairly fast.

Regards,
Martin