Learning python networking

[Chris Angelico]

On Thu, Jan 9, 2014 at 9:27 AM, Paul Pittlerson <menkomigen6 at gmail.com> wrote:
> I'm trying to learn about socket, how to create and handle connections in python.

Awesome! I *love* socket networking. (Really. It's awesome. I've
written a couple of MUD servers and quite a few MUD clients.)

> This is the best I could come up with on my own, please take a look and give me critique:
> Server script:
> http://pastebin.com/KtapYfM0
>
> Client script:
> http://pastebin.com/t4dYygmX

On this list we prefer in-line code as part of the post, but these are
a little long. Posting like this creates a dependency on that web
site, so a lot of people either can't or won't see your code.

> How to run it:
> I open 3 terminals, in one I start the server script, and enter into it something like 'accept 2'
>
> Then in the two other terminals I start client scripts which will "connect to the server" so to speak.
>
> Now I can communicate between them by sending messages, and exit the whole operation by typing 'exit' into the server.

>From what I'm seeing in that code, all communication is one-way,
right? The server sends to the clients, nothing comes back?

> Is the code overly complicated? More precisely: is there a more elegant and simple way to achieve the same thing? Some people have mentioned things like twisted and asyncore, but I don't know anything about them. If it turns out this kind of concept is very straight forward to set up in either of those I would be interested in sample code.

Those sorts of frameworks would be helpful if you need to scale to
infinity, but threads work fine when it's small.

> I'm specifically looking into this kind of communication because I want to make a small multiplayer game.

Absolutely! The thing to look at is MUDs and chat servers. Ultimately,
a multiplayer game is really just a chat room with a really fancy
front end.

So, some comments on your code.

The server shouldn't require interaction at all. It should accept any
number of clients (rather than getting the exact number that you
enter), and drop them off the list when they're not there. That's a
bit of extra effort but it's hugely beneficial.

One extremely critical point about your protocol. TCP is a stream -
you don't have message boundaries. You can't depend on one send()
becoming one recv() at the other end. It might happen to work when you
do one thing at a time on localhost, but it won't be reliable on the
internet or when there's more traffic. So you'll need to delimit
messages; I recommend you use one of two classic ways: either prefix
it with a length (so you know how many more bytes to receive), or
terminate it with a newline (which depends on there not being a
newline in the text).

Another rather important point, in two halves. You're writing this for
Python 2, and you're writing with no Unicode handling. I strongly
recommend that you switch to Python 3 and support full Unicode. Your
current code might work fine so long as everyone uses the same
codepage, but then you'll meet someone from Greece or Russia and
they'll be typing in gibberish because you're decoding it wrongly.
(You can get that sort of thing even without crossing country borders,
but then it's tempting to just say "Don't use funny characters"; but
the problem isn't funny characters, of which there aren't any (I
swear, Spike Milligan is *so* not funny... okay, that's not true), but
of encodings and character sets.) Using Python 3.4 (which isn't yet
stable, but you can download betas) also gives you an asyncio module,
but I'd leave that aside for the moment; first figure out threading,
it's likely to be easier.

So here's how I'd structure a program like this.

# Server
bind socket to port and listen on it
while True:
    accept socket
    spawn thread

thread:
    register self with list of connected clients
    while socket connected:
        receive data
        try to parse a message out of data - if not, buffer it
        handle message, eg by sending to all connected clients
    unregister self from connected client list


# Client
connect to server
start thread
while True:
    accept input from user
    act on input, which might 'break'
shut down socket cleanly

thread:
    receive data from socket
    try to parse a message out of data, as with the server
    # you might even be able to use the same code for both
    handle message, eg by displaying to screen



That's a threaded system. It emphasizes that the server is doing one
"thing" for each connected client (listening for incoming socket
data), plus one more "thing" (listening for new clients), and the
client is doing two "things" (listening for commands from the human,
and listening for messages from the server). Moving to an async I/O
system doesn't change that, it just runs it all on one thread. So do
whichever's easier to get your head around :)

Note, by the way, that it's helpful to distinguish "data" and "text",
even in pseudo-code. It's impossible to send text across a socket -
you have to send bytes of data. If you keep this distinction clearly
in your head, you'll have no problem knowing when to encode and when
to decode. For what you're doing here, for instance, I would packetize
the bytes and then decode into text, and on sending, I'd encode text
(UTF-8 would be hands-down best here) and then packetize. There are
other options but that's how I'd do it.

This is the Flight Level 200 overview of socket handling. I'm happy to
expand on any part that interests or confuses you (especially if it
does both at once).

The world is so much fun when you can wield multiple computers!

ChrisA