sending very large packets over the network

[Gary Herron]

Walker Lindley wrote:
> OK, I'm back with another networking question. I'm trying to seend 
> large amounts of information over TCP (the length of data being given 
> to send() is on the order of 16000 characters in length). 
> Unfortunately on the receiving end, the packets appear to be 
> truncated. So I wrote some code that continuously tries to send bigger 
> and bigger packets until it fails and noticed that it never fails at 
> the same length. I'm not even sure these two things are related, but 
> is there some undocumented (or documented and I missed it) maximum 
> size for data you can pass to send()?
For ethernet connections the size is often about 1500.  But the size 
depends on the underlying protocol, and even if you know the underlying 
protocol along the full route, you can't rely on packets that are 
received being the same as those that are sent.

TCP/IP is a *stream* connection.  What you are guaranteed is this:  All 
the bytes that are sent from one end will be received eventually on the 
other end, in the proper order.  (Or failing that, you will receive an 
error notification.)  No guarantee is made about the packet sizes on 
either end, and you can't even rely on the packets being the same in 
number or length on the two ends.

Your send code can try sending a packet of any size, but it must be 
prepared to examine the number of bytes actually sent, and retry with 
the remainder in a loop until all bytes are sent.    Similarly your 
receiving code must loop around the receive accepting whatever sized 
packets makes it through the connection.

In many TCP/IP connections, it seems that the packets received are 
one-for-one with the packets sent, but relying on this *IS AN ERROR* 
that will bite you.  It fails to across the internet (sometimes) and 
when (at least some) wireless cards are involved.

You may be better off using a package that knows all this and handles it 
properly.  Modules asyncore and asynchat are one possibility.


Gary Herron
>
> the sample code is as follows
> #server
> import socket
>
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.bind(("", 2000))
> s.listen(0)
> sock, addrinfo = s.accept()
> for i in range(2 ** 16):
>  length = int(sock.recv(16))
>  print "excpecting data of length:", length
>  data = sock.recv(length)
>  print "received data of length:", len(data)
>  print
> s.close()
> sock.close()
>
> #client
> import socket
> import time
>
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.bind(("", 2001))
> s.connect(("localhost", 2000))
> for i in range(2 ** 16):
>  packet = "h" * i
>  s.send("%16d" % len(packet))
>  print "attempting to send data of length:", len(packet)
>  tmp = s.send(packet)
>  print "actually sent data of length:", tmp
>  print
>  time.sleep(.001)
> s.close()
>
> i just put them in different files and ran them from the command line. 
> Any help or suggestions would be greatly appreciated. Thanks.
>
>
> -Walker
>
> -- 
> This e-mail is licensed under the Creative Commons 
> Attribution-NoDerivs 2.5 License. To view a copy of this license, 
> visit http://creativecommons.org/licenses/by-nd/2.5/ or send a letter 
> to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, 
> California, 94105, USA.