Opinions, please: Repository module

andy andy at eastonwest.co.uk
Sun Jan 5 17:34:51 EST 2003


Ok, I know I'm going out on a limb here: forum newbie who no-one knows ;-) ...

I'm working on a little project - for my kids, actually - and I had a need to 
code a module to provide a 'central network place to store miscellaneous 
stuff'.  No security or reliablility assurance is required either.  

I googled for an existing module, but found none simple enough (got lots of 
interesting stuff about distributed processing systems and so on though)...

Initially, I thought "ok, just use a shared directory".  Tried that, but the 
latency was *way* too severe, and unpredictable, even on my little 100mb 
network.

I realised I was going to have to bite the bullet and learn to use sockets.  I 
was off work for the christmas break, 9 days or so, and spent much of this 
time digging, until I was totally googled out.  

After a bit of a tough learning curve (things not working as I expected them 
to etc.) I now have a working 'prototype'.

So, what I want to know is - Is anyone interested in trying it out? I'd like 
some (hopefully constructive) comments, but not really along the lines of 
'you should have used x, y or z library bacause lah lah lah', more on doing 
what I'm doing better.

It uses sockets, and is loosely based on an old chat-server example by J. 
Strout (www.strout.net), so it uses a round-robin approach, with the sockets 
in non-blocking mode.  I specifically wanted to code this myself, pig-headed 
that I am, to learn a bit of the practicalities of socket programming; yes I 
know I could have used SocketServer, Twisted or any number of other modules 
hanging round there in cyberspace.  However, as I said, I wanted to know how 
it worked from the /inside/.

My solution is not that elegant yet (I'm working on that though) but it does 
seem to work ok; it's as fast as I need it and it takes minimal cpu - hardly 
noticeable on my Linux PII/300.

What it does:

Sets up a socket.
Listens on it.
Sets it to non-blocking mode.
Loops forever:
	checks for any new connections;
		stores them in a connection list.
	iterates the connetction list:
		checks for any incoming data on each connection
			processes each request and returns a result

Requests come in as a clear-text dictionary:
	{"request":"...","keys":{"k":v...}}

	These are evaluated (for convenience) by eval().  Yes, I know this is risky,
	but it' easy, ok.

Response returned as the original incoming key with 'comments':
	# successful request
	'{"request":"...","response":"ok","keys":{"k":v...}}  '
	- or -
	# failed request
	'{"request":"...","response":"no","reason":"error string","keys":{"k":v...}}'

So far only two requests are supported:

	"set" - sets key values
	"query" - queries key values

The key values are stored in a dictionary by the server.

I'm going to have to clean the code up a lot before I can post it.  

I just /bet/ someone is going to tell me now that there's already something 
out there which does exactly this :-(

Anyway, anyone interested?

-andyj





More information about the Python-list mailing list