object references/memory access

[dlomsak]

Martin v. Löwis wrote:
> > I guess now I'd like to know what are good practices in general to get
> > better results with sockets on the same local machine. I'm only
> > instantiating two sockets total right now - one client and one server,
> > and the transfer is taking 15 seconds for only 8.3MB.
>
> It would be good if you had showed the code that does that. It is hard
> for us to guess what programming error you have made.
>
> As you won't show code, I will. Please try the attached cli.py and
> server.py on your machine, and report the timing. On my machine, I get
>
> 0.00105595588684 0.076632976532 8300000
>
> which means I can transmit 8.3MB in 76ms, which is a lot less than
> 15s.
>
> My guess is that you sum up the incoming data with
>
>   total_data += received_data
>
> That is O(n**2).
>
> Regards,
> Martin
>
> import socket,time,cStringIO
>
> t1 = time.time()
> s = socket.socket()
> s.connect(('localhost', 8989))
> t2 = time.time()
> storage = cStringIO.StringIO()
> while True:
>     data = s.recv(1024)
>     if not data:
>         break
>     storage.write(data)
> result = storage.getvalue()
> t3 = time.time()
>
> print t2-t1,t3-t2,len(result)
>
> import socket
>
> data = ' '*8300000
> s = socket.socket()
> s.bind(('', 8989))
> s.listen(10)
> while True:
>     s1, peer = s.accept()
>     print s1,peer
>     s1.send(data)
>     s1.close()

I would have put my code up if it were here but it is on my machine at
work which I can't touch until Monday. I know people tend to like to
see code when you're asking for help but it is not available to me
right now so I apologize. You are right though, I believe I made the
mistake of using += to sum the data up and I had never considered the
fact that the runtime of that approach is O(n^2).  I am willing to bet
that this was my major shortcoming and you just solved my problem. I
bet the reason that jacking up the socket.recv size is because it took
fewer concatenations. I'll give an official report tomorrow on weather
or not that was the fix but I am very convinced that you got it and
that I won't have to step around the socket transmission.

Thanks a lot Martin and also to the others who responded.