Buffer size when receiving data through a socket?

[Gabriel Genellina]

En Mon, 16 Jun 2008 21:21:35 -0300, John Salerno <johnjsal at NOSPAMgmail.com> escribió:

> I wrote some pretty basic socket programming again, but I'm still confused about what's happening with the buffer_size variable. Here are the server and client programs:
>
> --------------
>
> from socket import *
>
> host = ''
> port = 51567
> address = (host, port)
> buffer_size = 1024
>
> server_socket = socket(AF_INET, SOCK_STREAM)
> server_socket.bind(address)
> server_socket.listen(5)
>
> while True:
>     print 'waiting for connection...'
>     client_socket, client_address = server_socket.accept()
>     print '...connected from:', client_address
>
>     while True:
>         data = client_socket.recv(buffer_size)
>         if not data:
>             break
>         client_socket.send('%s %s' % ('You typed:', data))
>
>     client_socket.close()
>
> server_socket.close()
>
> ------------
>
> from socket import *
>
> host = 'localhost'
> port = 51567
> address = (host, port)
> buffer_size = 1024
>
> client_socket = socket(AF_INET, SOCK_STREAM)
> client_socket.connect(address)
>
> while True:
>     data = raw_input('> ')
>     if not data:
>         break
>     client_socket.send(data)
>     data = client_socket.recv(buffer_size)
>     if not data:
>         break
>     print data
>
> client_socket.close()
>
> ---------------
>
> I tried changing buffer_size to 10 and I got this output:
>
> john at john-laptop:~$ python myclient.py
>> hello
> You typed:
>> something
>  hello
>> this is a long string
> You typed:
>> why doesn't this work right
>  something
>>
> john at john-laptop:~$
>
> My first question is, isn't buffer_size the number of bytes being sent at one time? If so, why doesn't 'hello' get printed after the server returns the data to the client? Isn't 'hello' just 5 bytes?

Both programs say recv(buffer_size) - buffer_size is the maximum number of bytes to be RECEIVED, that is, READ. recv will return at most buffer_size bytes. It may return less than that, even if the other side sent the data in a single operation.
Note that most of the time you want to use the sendall() method, because send() doesn't guarantee that all the data was actually sent. <http://docs.python.org/lib/socket-objects.html>

> Secondly, how is it working that once I type in a new string (e.g. 'something') and then the server returns data to the client, it prints the *previous* string, (i.e. 'hello')? Wouldn't the data variable get overwritten with the value, or is the value being stored somewhere else at this point?

Yes, it is stored in an intermediate buffer until you read it. You typed "hello" and sent it, the server replied with the string "You typed: hello"; the OS stores it. You read only 10 bytes "You typed:", the remaining are still in the buffer. Next round: you type something, the server replies, you read the remaining bytes from the original reply, and so on...

(Note that in this particular configuration, the client will fill its buffer at some time: because the server sends at least 11 bytes each round, but the client reads at most 10 bytes, so the client is always behind the server...)

-- 
Gabriel Genellina