socket.makefile() buggy?

[Frank Swarbrick]

ahlongxp wrote:
> On Jul 8, 9:54 pm, Steve Holden <s... at holdenweb.com> wrote:
>> That's a pretty pejorative subject line for someone who's been
>> programming Python [guessing by the date of your first post] for about a
>> month.
>>
> I have to admit it that I'm quite  a newbie programmer.
>> Perhaps "Incomprehensible behavior from socket.makefile()", or "I have
>> written a buggy network application"? That would at least show that you
>> are considering the possibility you yourself are the cause of the
>> problem ;-)
>>
> Thanks. I'll be more careful about my words.
> but I did show some possibility myself is the cause of the problem.
> Is there any chance you missed the "?" and "guess" at the same time.
> 
>> Python has been around for a long time, so you should ask yourself how
>> likely it is that you would be the first to discover such a fundamental
>> flaw?
> very low but not zero.
> Miracle happens.
> 
>> I'd be very surprised if someone doesn't point you at an article
>> on "how to ask smart questions", but I will content myself with that
>> oblique reference.
>>
>>
> The big problem is that, I didn't know such a person like you before.
> It's my misfortune.
> 
> 
>> The big problem here seems to be that your server is closing the socket
>> after reading 99 bytes, but the client is actually sending 120. If you
>> change the "99" in the server to "120" you will see that the client
>> works when it uses the makefile().read(). I can't be bothered to debug
>> the exact reason why the 21 bytes of buffered data doesn't cause a
>> problem with the recv() version, but I wouldn't regard the error you are
>> getting as a bug in Python, rather as a bug in your application.
>>
> I don't know the underlying details of networking.
> But with recv()  version I can get things done as expected while
> makefile()
> version can't.
> I guess it's not that unreasonable to make a guess like I did.
>> The underlying cause of all this appears to be your failure to
>> understand that network protocols should ideally allow the endpoints to
>> determine when a complete message has been transmitted, and to consume
>> all transmitted data.
>>
>> You can't just ignore some or all of a client request and expect it not
>> to cause problems.
> Back to the problem, I make it working  by adding a sleep(0.5) just
> before the
> server closes the connection.
> Actually this is Hendrik van Rooyen's idea.
> And luckily I got a another lesson
> http://groups.google.com/group/comp.lang.python/browse_thread/thread/9c5ad2608f4c80d5/4302772dfe27d8f1#4302772dfe27d8f1

It would be nice to know just exactly what you are trying to achieve. 
Sure, adding the sleep(0.5) works, but I think that is just a 
work-around and doesn't really fix the underlying problem.  The 
underlying problem, of course, being that the server is closing the 
connection while the client is still trying to send data.

Seems to me what you really want to do is either have the send and the 
recv in different threads, or use non-blocking sockets and select.  Or 
maybe you don't even need to use select necessarily.  Seems like you 
could use non-blocking sockets and then attempt a recv after each send, 
in order to see if the response is ready.  You want to use a 
non-blocking socket so that if there's no response ready the recv will 
not block waiting for one.  Maybe after your last send you'd want to put 
it back in blocking mode prior to doing the final recv.  Or something 
that that!  :-)

Hope that gives you a start, or at least some better questions to ask.

Frank