From pa.basso at gmail.com  Thu Feb 17 10:42:50 2011
From: pa.basso at gmail.com (Paolo Basso)
Date: Thu, 17 Feb 2011 10:42:50 +0100
Subject: [execnet-dev] Little question
Message-ID: <AANLkTinmBet3upKAvRDpDK+Z=q3n1qPVfLoC27hyZzQH@mail.gmail.com>

Hi,

I'm new to the list and to execnet.
I'm trying to figure out if it is possible to send n items to k channels
(where n>k) so that, as soon as the first of the k channels sends back its
result, immediatly takes the next item (k+1) in the list and so on.
Can anybody illuminate me?

Thanks,

Paolo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20110217/7b7716d7/attachment.html>

From holger at merlinux.eu  Thu Feb 17 15:34:15 2011
From: holger at merlinux.eu (holger krekel)
Date: Thu, 17 Feb 2011 15:34:15 +0100
Subject: [execnet-dev] Little question
In-Reply-To: <AANLkTinmBet3upKAvRDpDK+Z=q3n1qPVfLoC27hyZzQH@mail.gmail.com>
References: <AANLkTinmBet3upKAvRDpDK+Z=q3n1qPVfLoC27hyZzQH@mail.gmail.com>
Message-ID: <20110217143415.GD30557@trillke.net>

Hi Paolo,

have you looked at  
http://codespeak.net/execnet/example/test_multi.html 
and the example section in general yet?

best,
holger

On Thu, Feb 17, 2011 at 10:42 +0100, Paolo Basso wrote:
> Hi,
> 
> I'm new to the list and to execnet.
> I'm trying to figure out if it is possible to send n items to k channels
> (where n>k) so that, as soon as the first of the k channels sends back its
> result, immediatly takes the next item (k+1) in the list and so on.
> Can anybody illuminate me?
> 
> Thanks,
> 
> Paolo

> _______________________________________________
> execnet-dev mailing list
> execnet-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/execnet-dev


-- 


From pa.basso at gmail.com  Thu Feb 17 15:56:23 2011
From: pa.basso at gmail.com (Paolo Basso)
Date: Thu, 17 Feb 2011 15:56:23 +0100
Subject: [execnet-dev] Little question
In-Reply-To: <20110217143415.GD30557@trillke.net>
References: <AANLkTinmBet3upKAvRDpDK+Z=q3n1qPVfLoC27hyZzQH@mail.gmail.com>
	<20110217143415.GD30557@trillke.net>
Message-ID: <AANLkTimvzed9Y_z7P6ULFUs0DtEbMzN=S7xCMLDNAn=P@mail.gmail.com>

Yes, I looked there but, as far as I can understand, there isn't an example
which satisfies my question.
Maybe I'm wrong but what if I'm in a case like this:

- a cluster of 3 computers (2 cores each)
- I have to evaluate 8 items

The most natural thing should be to create 3 different gatways and 2 channel
for each gatway.
But if I have to assign "a priori" the items to the channels how can I do
this?
I mean, I can see the possibility to create 3 channels for the first two
gatways and 2 for the last one (so that I have 8 items and 8 channels) and
get the results with "mch.receive_each()". But who told me that is better to
subdivide the channels like this?
I would prefere something like "mch.sandcallback()" which automatically
sends back a new item to the first free channel till the end of the items,
independently of the number of channels.

Thanks,

Paolo



2011/2/17 holger krekel <holger at merlinux.eu>

> Hi Paolo,
>
> have you looked at
> http://codespeak.net/execnet/example/test_multi.html
> and the example section in general yet?
>
> best,
> holger
>
> On Thu, Feb 17, 2011 at 10:42 +0100, Paolo Basso wrote:
> > Hi,
> >
> > I'm new to the list and to execnet.
> > I'm trying to figure out if it is possible to send n items to k channels
> > (where n>k) so that, as soon as the first of the k channels sends back
> its
> > result, immediatly takes the next item (k+1) in the list and so on.
> > Can anybody illuminate me?
> >
> > Thanks,
> >
> > Paolo
>
> > _______________________________________________
> > execnet-dev mailing list
> > execnet-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/execnet-dev
>
>
> --
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20110217/11ebc96b/attachment.html>

From holger at merlinux.eu  Thu Feb 17 16:26:32 2011
From: holger at merlinux.eu (holger krekel)
Date: Thu, 17 Feb 2011 16:26:32 +0100
Subject: [execnet-dev] Little question
In-Reply-To: <AANLkTimvzed9Y_z7P6ULFUs0DtEbMzN=S7xCMLDNAn=P@mail.gmail.com>
References: <AANLkTinmBet3upKAvRDpDK+Z=q3n1qPVfLoC27hyZzQH@mail.gmail.com>
	<20110217143415.GD30557@trillke.net>
	<AANLkTimvzed9Y_z7P6ULFUs0DtEbMzN=S7xCMLDNAn=P@mail.gmail.com>
Message-ID: <20110217152632.GE30557@trillke.net>

Hi Paolo,

is the attached example roughly what you need? 
It opens 2 gateways per dual-core machine, conceptually.
That's easier than multiplexing which we could implement
if neccesary, like with a 256 core machine or so :)

cheers,
holger

On Thu, Feb 17, 2011 at 15:56 +0100, Paolo Basso wrote:
> Yes, I looked there but, as far as I can understand, there isn't an example
> which satisfies my question.
> Maybe I'm wrong but what if I'm in a case like this:
> 
> - a cluster of 3 computers (2 cores each)
> - I have to evaluate 8 items
> 
> The most natural thing should be to create 3 different gatways and 2 channel
> for each gatway.
> But if I have to assign "a priori" the items to the channels how can I do
> this?
> I mean, I can see the possibility to create 3 channels for the first two
> gatways and 2 for the last one (so that I have 8 items and 8 channels) and
> get the results with "mch.receive_each()". But who told me that is better to
> subdivide the channels like this?
> I would prefere something like "mch.sandcallback()" which automatically
> sends back a new item to the first free channel till the end of the items,
> independently of the number of channels.
> 
> Thanks,
> 
> Paolo
> 
> 
> 
> 2011/2/17 holger krekel <holger at merlinux.eu>
> 
> > Hi Paolo,
> >
> > have you looked at
> > http://codespeak.net/execnet/example/test_multi.html
> > and the example section in general yet?
> >
> > best,
> > holger
> >
> > On Thu, Feb 17, 2011 at 10:42 +0100, Paolo Basso wrote:
> > > Hi,
> > >
> > > I'm new to the list and to execnet.
> > > I'm trying to figure out if it is possible to send n items to k channels
> > > (where n>k) so that, as soon as the first of the k channels sends back
> > its
> > > result, immediatly takes the next item (k+1) in the list and so on.
> > > Can anybody illuminate me?
> > >
> > > Thanks,
> > >
> > > Paolo
> >
> > > _______________________________________________
> > > execnet-dev mailing list
> > > execnet-dev at codespeak.net
> > > http://codespeak.net/mailman/listinfo/execnet-dev
> >
> >
> > --
> >

-- 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: multi_channel.py
Type: text/x-python
Size: 659 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20110217/194767f1/attachment.py>

From holger at merlinux.eu  Thu Feb 17 17:00:36 2011
From: holger at merlinux.eu (holger krekel)
Date: Thu, 17 Feb 2011 17:00:36 +0100
Subject: [execnet-dev] refined taskserver with better termination Re: Little
	question
In-Reply-To: <20110217152632.GE30557@trillke.net>
References: <AANLkTinmBet3upKAvRDpDK+Z=q3n1qPVfLoC27hyZzQH@mail.gmail.com>
	<20110217143415.GD30557@trillke.net>
	<AANLkTimvzed9Y_z7P6ULFUs0DtEbMzN=S7xCMLDNAn=P@mail.gmail.com>
	<20110217152632.GE30557@trillke.net>
Message-ID: <20110217160036.GF30557@trillke.net>

Paolo,

i refined the example and put it into the repo:

    https://bitbucket.org/hpk42/execnet/src/527c8b199c9d/doc/example/taskserver.py

please let me know if the scheme works for you and if there remain questions.

cheers,

holger

On Thu, Feb 17, 2011 at 16:26 +0100, holger krekel wrote:
> Hi Paolo,
> 
> is the attached example roughly what you need? 
> It opens 2 gateways per dual-core machine, conceptually.
> That's easier than multiplexing which we could implement
> if neccesary, like with a 256 core machine or so :)
> 
> cheers,
> holger
> 
> On Thu, Feb 17, 2011 at 15:56 +0100, Paolo Basso wrote:
> > Yes, I looked there but, as far as I can understand, there isn't an example
> > which satisfies my question.
> > Maybe I'm wrong but what if I'm in a case like this:
> > 
> > - a cluster of 3 computers (2 cores each)
> > - I have to evaluate 8 items
> > 
> > The most natural thing should be to create 3 different gatways and 2 channel
> > for each gatway.
> > But if I have to assign "a priori" the items to the channels how can I do
> > this?
> > I mean, I can see the possibility to create 3 channels for the first two
> > gatways and 2 for the last one (so that I have 8 items and 8 channels) and
> > get the results with "mch.receive_each()". But who told me that is better to
> > subdivide the channels like this?
> > I would prefere something like "mch.sandcallback()" which automatically
> > sends back a new item to the first free channel till the end of the items,
> > independently of the number of channels.
> > 
> > Thanks,
> > 
> > Paolo
> > 
> > 
> > 
> > 2011/2/17 holger krekel <holger at merlinux.eu>
> > 
> > > Hi Paolo,
> > >
> > > have you looked at
> > > http://codespeak.net/execnet/example/test_multi.html
> > > and the example section in general yet?
> > >
> > > best,
> > > holger
> > >
> > > On Thu, Feb 17, 2011 at 10:42 +0100, Paolo Basso wrote:
> > > > Hi,
> > > >
> > > > I'm new to the list and to execnet.
> > > > I'm trying to figure out if it is possible to send n items to k channels
> > > > (where n>k) so that, as soon as the first of the k channels sends back
> > > its
> > > > result, immediatly takes the next item (k+1) in the list and so on.
> > > > Can anybody illuminate me?
> > > >
> > > > Thanks,
> > > >
> > > > Paolo
> > >
> > > > _______________________________________________
> > > > execnet-dev mailing list
> > > > execnet-dev at codespeak.net
> > > > http://codespeak.net/mailman/listinfo/execnet-dev
> > >
> > >
> > > --
> > >
> 
> -- 

> 
> import execnet
> 
> group = execnet.Group()
> group.makegateway()
> group.makegateway()
> group.makegateway()
> 
> def process_item(channel):
>     import time, random
>     channel.send("ready")
>     for x in channel:
>         time.sleep(random.randrange(3))
>         channel.send(x*10)
> 
> mch = group.remote_exec(process_item)
> q = mch.make_receive_queue()
> tasks = range(10)
> while tasks:
>     channel, item = q.get()
>     if item != "ready":
>         print "other side %s returned result %r" % (channel.gateway.id, item)
>     task = tasks.pop()
>     channel.send(task)
>     print "sent task %r to %s" % (task, channel.gateway.id)
> 
> print "got all results, terminating"
> group.terminate()


-- 


From samy.vilar at gmail.com  Sat Feb 26 00:31:53 2011
From: samy.vilar at gmail.com (Samy Vilar)
Date: Fri, 25 Feb 2011 18:31:53 -0500
Subject: [execnet-dev] help
Message-ID: <AANLkTikN5r=_bT=NrOCaTypd_Hn2d7U07mBrj_BTt9t2@mail.gmail.com>

This has being driving me nuts for a while, I have a small network about
half a dozen machines, all with python and third party modules installed
in custom locations, now loading the appropriate python interpreter at each
machines was trivial but every time I try to import a module I keep getting
an error 'No module named ...', Im assuming, since execnet ssh with notty
into a machine, bashrc or bashprofile isn't loaded and so the paths aren't
being set, I've also tried adding the paths by remote_exec('import sys;
sys.path.append(...)') and still nothing, I've even tried using chdir to
point to my site-packages dir and still nothing, please help me? I'm
currently using execnet version 1.0.9
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/execnet-dev/attachments/20110225/2a712188/attachment.html>