socket.makefile() buggy?

[Steve Holden]

That's a pretty pejorative subject line for someone who's been
programming Python [guessing by the date of your first post] for about a
month.

Perhaps "Incomprehensible behavior from socket.makefile()", or "I have
written a buggy network application"? That would at least show that you
are considering the possibility you yourself are the cause of the
problem ;-)

Python has been around for a long time, so you should ask yourself how
likely it is that you would be the first to discover such a fundamental
flaw? I'd be very surprised if someone doesn't point you at an article
on "how to ask smart questions", but I will content myself with that
oblique reference.

ahlongxp wrote:
> socket.makefile() may lose data when "connection reset by peer".
> and socket.recv() will never lose the data.
> 
> change the "1" to "0" in the client code to see the difference.
> 
> confirmed on both windows and linux.
> 
> so I guess there is a problem with makefile().
> 
> # Echo server program
> import socket
> 
> HOST = ''                 # Symbolic name meaning the local host
> PORT = 50007              # Arbitrary non-privileged port
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
> s.bind((HOST, PORT))
> s.listen(1)
> while(1):
>     conn, addr = s.accept()
>     print 'Connected by', addr
>     conn.settimeout(1)
>     toread = 99
>     retrytime = 0
>     reads = 0
>     while reads < toread and retrytime < 10:
>         try:
>             data = conn.recv(min(32,toread-reads))
>             if not data: continue
>             print data
>             reads += len(data)
>         except:
>             retrytime += 1
>             print "timeout %d" % retrytime
>             continue
>     #conn.shutdown(socket.SHUT_RD)#no more read
>     if reads == toread:
>         conn.send("OK")
>     else:
>         conn.send("NOT OK")
>     conn.close()
> 
> # Echo client program
> import socket
> 
> HOST = 'localhost'    # The remote host
> PORT = 50007              # The same port as used by the server
> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> s.connect((HOST, PORT))
> for i in range(12):
>     print "time %d" % i
>     try:
>         s.send('0123456789')
>     except socket.error, e:
>         print "socket error:", e
>         break
>     #data = s.recv(1024)
>     #print "data %d" %i, data
> #s.shutdown(socket.SHUT_WR)#no more write
> 
> '''
> try changing 1 to 0.
> '''
> if 1:
>     data=s.recv(1024)
> else:
>     rf = s.makefile("rb")
>     data = rf.read()
>     rf.close()
> s.close()
> print 'Received', repr(data)
> 
> 
The big problem here seems to be that your server is closing the socket
after reading 99 bytes, but the client is actually sending 120. If you
change the "99" in the server to "120" you will see that the client
works when it uses the makefile().read(). I can't be bothered to debug
the exact reason why the 21 bytes of buffered data doesn't cause a
problem with the recv() version, but I wouldn't regard the error you are
getting as a bug in Python, rather as a bug in your application.

The underlying cause of all this appears to be your failure to
understand that network protocols should ideally allow the endpoints to
determine when a complete message has been transmitted, and to consume
all transmitted data.

You can't just ignore some or all of a client request and expect it not
to cause problems. If you look at some of the better-known TCP-based
application protocols you will see they take pains to ensure that
clients and servers can deterministically parse each others' messages.

In summary, a better protocol design will cause you rather less grief
and a little more humility will result in less acidity from crabby old
geeks like me.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd           http://www.holdenweb.com
Skype: holdenweb      http://del.icio.us/steve.holden
--------------- Asciimercial ------------------
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
----------- Thank You for Reading -------------