Lightwight socket IO wrapper

[James Harris]

"Akira Li" <4kir4.1i at gmail.com> wrote in message 
news:mailman.18.1442804862.28679.python-list at python.org...
> "James Harris" <james.harris.1 at gmail.com> writes:
> ...
>> There are a few things and more crop up as time goes on. For example,
>> over TCP it would be helpful to have a function to receive a specific
>> number of bytes or one to read bytes until reaching a certain
>> delimiter such as newline or zero or space etc.
>
> The answer is sock.makefile('rb') then `file.read(nbytes)` returns a
> specific number of bytes.

Thanks, I hadn't seen that. Now I know of it I see references to it all 
over the place but beforehand it was in hiding....

It is exactly the type of convenience wrapper I was expecting Python to 
have but expected it to be in another module. It looks as though it will 
definitely cover some of the issues I had.

> `file.readline()` reads until newline (b'\n') There is Python Issue:
> "Add support for reading records with arbitrary separators to the
> standard IO stack"
>  http://bugs.python.org/issue1152248
> See also
>  http://bugs.python.org/issue17083
>
> Perhaps, it is easier to implement read_until(sep) that is best suited
> for a particular case.

OK.

...

>> When sending it would be good to just say to send a bunch of bytes 
>> but
>> know that you will get told how many were sent (or didn't get sent) 
>> if
>> it fails. Sock.sendall() doesn't do that.
>
> sock.send() returns the number of bytes sent that may be less than 
> given.
> You could reimplement sock.sendall() to include the number of bytes
> successfully sent in case of an error.

I know. As mentioned, I wondered if there were already such functions to 
save me using my own.

>> I thought UDP would deliver (or drop) a whole datagram but cannot 
>> find
>> anything in the Python documentaiton to guarantee that. In fact
>> documentation for the send() call says that apps are responsible for
>> checking that all data has been sent. They may mean that to apply to
>> stream protocols only but it doesn't state that. (Of course, UDP
>> datagrams are limited in size so the call may validly indicate
>> incomplete transmission even when the first part of a big message is
>> sent successfully.)
>>
>> Receiving no bytes is taken as indicating the end of the
>> communication. That's OK for TCP but not for UDP so there should be a
>> way to distinguish between the end of data and receiving an empty
>> datagram.
>
> There is no end of communication in UDP and therefore there is no end 
> of
> data. If you've got a zero bytes in return then it means that you've
> received a zero length datagram.
>
> sock.recvfrom() is a thin wrapper around the corresponding C
> function. You could read any docs you like about UDP sockets.
> 
> http://stackoverflow.com/questions/5307031/how-to-detect-receipt-of-a-0-length-udp-datagram

As mentioned to Dennis just now, I would prefer to write code to conform 
with the documented behaviour of Python and its libraries, as long as 
they were known to be reliable implementations of what was documented, 
of course.

I agree with what you say. A zero-length UDP datagram should be possible 
and not indicate end of input but is that guaranteed and portable? 
(Rhetorical.)  It seems not. Even the Linux man page for recv says: "If 
no  messages  are  available  at  the  socket, the receive calls wait 
for a message to arrive, unless the socket is nonblocking...." In that 
case, of course, what it defines as a "message" - and whether it can be 
zero length or not - is not stated.

>> The recv calls require a buffer size to be supplied which is a
>> technical detail. A Python wrapper could save the programmer dealing
>> with that.
>
> It is not just a buffer size. It is the maximum amount of data to be
> received at once i.e., sock.recv() may return less but never more.

My point was that we might want to request the entire next line or next 
field of input and not know a maximum length. *C* programmers are used 
to giving buffers fixed sizes often because then they can avoid fiddling 
with memory management but Python normally does that for us. I was 
suggesting that the thin wrapper around the socket recv() call is too 
thin! The makefile() approach that you mentioned seems more Pythonesque, 
though.

> You could use makefile() and read() if recv() is too low-level.

Yes.

James