[Python-Dev] The socket HOWTO

[exarkun at twistedmatrix.com]

On 5 Jun, 10:35 pm, martin at v.loewis.de wrote:
>>First, Twisted doesn't always use the BSD sockets API; the Windows 
>>IOCP
>>reactor, especially, starts off with the socket() function, but things
>>go off in a different direction pretty quickly from there.
>
>Hmm. Are you saying it doesn't use listen, connect, bind, send, recv?
>To me, that's the core of BSD sockets. I can understand it doesn't
>use select(2).

Yes, that's correct.  Those aren't the best APIs to use on Windows, so 
they aren't necessarily used on Windows.
>>So it's
>>perfectly fine to introduce yourself to networking via Twisted, and 
>>many
>>users have done just that.  If you're using it idiomatically, you 
>>should
>>never encounter a socket object or file descriptor poking through the
>>API anywhere.
>
>And that's all fine. I still claim that you have to *understand*
>sockets in order to use it properly. By this, I mean stuff like
>"what is a TCP connection? how is it established?", "how is UDP
>different from TCP?", "when data arrives, what layers of software
>does it go through?", "what is a port number?", etc.

These may be good things to understand.  The current socket howto 
doesn't explain them, though.
>>Second, it makes me a little sad that it appears to be folk wisdom 
>>that
>>Twisted is only for servers.  A lot of work has gone into making it
>>equally appropriate for clients.  This is especially true if your 
>>client
>>has a GUI, where Twisted is often better than a protocol-specific
>>library, which may either be blocking or have its own ad-hoc event 
>>loop.
>
>I think that's because many of the problems that Twisted solves don't
>exist in many of the client applications - in particular, you often
>don't have many simultaneous connections. GUI apps may be the
>interesting special case, but I expect that people dealing with these
>rather use separate threads.

On the contrary, many of the problems do exist in client applications 
(every time I have to use virt-manager I want to throw it out a window).

Some people certainly would rather use threading, but that doesn't say 
anything about whether Twisted solves problems relevant to clients, only 
about the fact that a lot of people like to use threads.
>>I don't have an opinion on the socket HOWTO per se, only on the
>>possibility of linking to Twisted as an alternate implementation
>>mechanism.  It really would be better to say "go use Twisted rather 
>>than
>>reading any of the following" than "read the following, which will 
>>help
>>you understand Twisted".
>
>Wouldn't you agree that Twisted is very difficult to learn, and thus
>much heavier than sockets? And I don't blame the Twisted API for that,
>but rather the mental model of overlapping activities that people have
>severe problems with.

This discussion has a significant problem, in taking "Twisted" as a 
monolithic all-or-nothing entity.  Restricting the scope to merely the 
lowest-level socket replacement APIs - ie, the bare TCP, UDP, etc 
functionality - no, Twisted is not very difficult to learn.  Expanding 
the scope to include the higher level functionality, it is much easier 
to learn than reimplementing line parsing and concurrency and so forth.

However, does that really have anything to do with improving the socket 
howto?  The Python documentation can include a clear explanation of what 
functionality the socket module provides - without forcing you to read 
Stevens _or_ use Twisted, but it can still refer you to both, since it 
is very likely that you'll need at least one of them in addition to the 
socket module.

Jean-Paul