client/server design and advice

[bruce]

hi irmen...

happened to come across this post. haven't looked at pyro. regarding your
'work packets' could these essentially be 'programs/apps' that that are
requested by the client apps, and are then granted by the dispatch/server
app?

i'm considering condor (univ of wisconsin) but am curious as to if pyro
might also work.

i'm looking to create a small distributed crawling app for crawling/scraping
of targeted websites....

thanks


-----Original Message-----
From: python-list-bounces+bedouglas=earthlink.net at python.org
[mailto:python-list-bounces+bedouglas=earthlink.net at python.org]On Behalf
Of Irmen de Jong
Sent: Friday, December 01, 2006 10:05 AM
To: python-list at python.org
Subject: Re: client/server design and advice


TonyM wrote:

> Lastly, as far as the networking goes, i have seen posts and such about
> something called Pyro (http://pyro.sourceforge.net) and wondered if
> that was worth looking into for the client/server interaction.

I'm currently busy with a new version of Pyro (3.6) and it already
includes a new 'distributed computing' example, where there is
a single dispatcher service and one or more 'worker' clients.
The clients request work 'packets' from the dispatcher and
process them in parallel.
Maybe this is a good starting point of your system?
Current code is available from Pyro's CVS repository.

--Irmen
--
http://mail.python.org/mailman/listinfo/python-list