Network programming in python

[Van Gale]

Rajarshi,

If this is more than an educational project you'll need to do some full
research on network programming in general.  Check out "Unix Network
Programming Vol. I" by W. Richard Stevens.  It is a classic and pretty much
regarded as a bible.

I don't know if there are any online resources that can explain server
models as good as that book.  It's important to understand the different
server models like single-process, single-process multiplexing, forking,
pre-forking, or threaded before you pick the path you want to go down in
Python.

Once you think you understand server models and their tradeoffs and socket
programming in general, you need to start researching the Python options.

At the lowest level is the socket module.  Read
http://www.mcmillan-inc.com/sock1.html for a quick HOWTO on using sockets at
the lowest level for client-server programming.

At a higher level things get a little murkier, which is why you need to
understand server models.  You have three choices here: 1) you can write
your own code directly on top of sockets, 2) you can use the SocketServer
module or 3) you can use the asyncore module.

SocketServer processes requests synchronously (single-process model), but
you can use the ForkingMixIn class to easily switch to a forking model, or
the ThreadingMixIn class to easily switch to a threading model.  The
SocketServer classes are pretty easy to use and much of the higher level
Python library modules use them.  They are not complicated to understand and
the Python library documentation and Fredrik Lundh's book "Python Standard
Library" cover them pretty well.

The asyncore module will probably provide better performance with its
single-process multiplexing model, but there are a few drawbacks.  First,
it's a more complicated model to grok, then add to that "the source is the
documentation".  You'll need to download Medusa from www.nightmare.com and
study that code.  Medusa gives you http, ftp, and xmlrpc classes on top of
asyncore.  To make things even more complicated, if you need to add
threading you'll need to download Zope and study how they implemented
threading on top of asyncore in the Zope ZServer.  The Nightmare web site
has links to many good resources about network programming.

As for how to "construct your messages" you can look at how http and ftp
messages are constructed by reading the appropriate modules in the Python
library or in medusa.  The Python library also has modules for gopher, imap,
pop, smtp, telnet, and nntp that you can use for ideas.  Or you can go
directly to the protocol documents written as RFC's.
http://www.cis.ohio-state.edu/Services/rfc/ is one place to browse RFC's.

If you are going to write clients in C/C++ I would recommend using a message
format like netstrings (see http://cr.yp.to/proto/netstrings.txt for info).
A Python implementation of netstrings can be found at
ftp://ftp.tummy.com/pub/tummy/Python/netstring/

Another option for constructing messages is to use SOAP or XML-RPC.  XML-RPC
is less complicated than SOAP, but more limited in capabilities.  Both SOAP
and XML-RPC will work on top of either SocketServer or asyncore.  As another
poster pointed out, these modules make network programming *really easy*
because it abstracts it to a simple Python function or method call.

Example:

Let's say you want to send two numbers from a client to your server, have
the server add them together and then return the result.

On the server you do this--

> import SOAP
> def add (x, y):  return x + y
> server = SOAP.SOAPServer(('localhost',9900))
> server.registerFunction(add)
> server.serve_forever ()

On the client you do this--

> import SOAP
> server = SOAP.SOAPProxy("http://localhost:9900")
> print server.add (23, 4)

Simple eh?  Unfortunately there are drawbacks :)  SOAP and XML-RPC messages
are constructed as valid XML documents, which have some limitations such as
binary data needing to be encoded as text (usually as base64) which can
expand message size.  This shouldn't be a problem if all your clients and
servers are on a LAN, but if you're passing megabyte sized files and have
clients on dial-up... uh not good.  (I think the Actzero SOAP.py is going to
support binary files as mime attachments, but it's not there yet).  Another
problem is performance.  Usually each function call opens a new connection
then closes it when the result is returned which can get expensive.

So, this is far from complete answer, but I hope I'm giving you an idea of
which way to go in your research!

Van