object references/memory access

[dlomsak]

Thanks for the responses folks. I'm starting to think that there is
merely an inefficiency in how I'm using the sockets. The expensive
part of the program is definitely the socket transfer because I timed
each part of the routine individually. For a small return, the whole
search and return takes a fraction of a second. For a large return (in
this case 21,000 records - 8.3 MB) is taking 18 seconds. 15 of those
seconds are spent sending the serialized results from the server to
the client. I did a little bit of a blind experiment and doubled the
bytes on the client's socket.recv line. This improved the rate of
transfer each time. The original rate when I was accepting 1024 bytes
per recv took 47 seconds to send the 8.3 MB result. By doubling this
size several times, I reduced the time to 18 seconds until doubling it
further produced diminishing results. I was always under the
impression that keeping the send and recv byte sizes around 1024 is a
good idea and I'm sure that jacking those rates up is a lousy way to
mitigate the transfer. It is also interesting to note that increasing
the bytes sent per socket.send on the server side had no visible
effect. Again, that was just a curious experiment.

What bothers me is that I am sure sending data over the local loopback
address should be blazing fast. 8.3 MB should be a breeze because I've
transferred files over AIM to people connected to the same router as
me and was able to send hundreds of megabytes in less than a two or
three seconds. With that said, I feel like something about how I'm
send/recv-ing the data is causing lots of overhead and that I can
avoid reading the memory directly if I can speed that up.

I guess now I'd like to know what are good practices in general to get
better results with sockets on the same local machine. I'm only
instantiating two sockets total right now - one client and one server,
and the transfer is taking 15 seconds for only 8.3MB. If you guys have
some good suggestions on how to better utilize sockets to transfer
data at the speeds I know I should be able to achieve on a local
machine, let me know what you do. At present, I find that using
sockets in python requires very few steps so I'm not sure where I
could really improve at this point.

Thanks for the replies so far, I really appreciate you guys
considering my situation and helping out.