[IPython-dev] finding the number of available engines

[Satrajit Ghosh]

hi min,

i've run into a related problem. i'm trying to run code on an engine that
can itself spawn new tasks, but i can't seem to execute function below on an
engine. it just doesn't return if you call this through the taskclient in
blocking mode.

it seems to block in the request for the task client and multiengine client.

cheers,

satra

-----
def get_ipy_clients():
    """Get ipython clients

    Returns client, taskclient and multiengineclient
    """
    ipyclient = None
    taskclient = None
    mecclient = None
    try:
        name = 'IPython.kernel.client'
        __import__(name)
        ipyclient = sys.modules[name]
    except ImportError:
        warn("Ipython kernel not found.  Parallel execution will be" \
                 "unavailable", ImportWarning)
    if ipyclient:
        try:
            taskclient = ipyclient.TaskClient()
            mecclient = ipyclient.MultiEngineClient()
        except Exception, e:
            if isinstance(e, ConnectionRefusedError):
                warn("No clients found, running serially for now.")
            if isinstance(e, ValueError):
                warn("Ipython kernel not installed")
    return ipyclient, taskclient, mecclient
------



On Wed, Jan 19, 2011 at 2:42 PM, MinRK <benjaminrk at gmail.com> wrote:

> mec.queue_status() returns a list of the form:
> [
>   (0, { 'pending' : "execute('a=5')", 'queue' : [ job1,job2,...] } ,
>   (1, {'pending' : 'None', 'queue' : [] },
>   ...
> ]
>
> In this case, engine 1 is idle.  I don't know why 1's pending is 'None'
> instead of None, that seems to be a bug.
>
> So you can see the idle engines with something like:
>
> def idle_engines(mec):
>     """return list of engine_ids corresponding to idle engines."""
>     qs = mec.queue_status()
>     engines = []
>     for e_id, status in qs:
>         if status['queue']:
>             continue
>         if not status['pending'] or status['pending'] == 'None':
>             engines.append(e_id)
>     return engines
>
> Which would return a list of engine_ids that are idle, the length of which
> would of course be the number of idle engines.
>
> -MinRK
>
>
> On Wed, Jan 19, 2011 at 10:39, Satrajit Ghosh <satra at mit.edu> wrote:
>
>> hi brian and min,
>>
>> i would like to do something like this:
>>
>> if num_engines_available() > 2:
>>     do_x
>> else:
>>     do_y
>>
>> in 0.10.1 series is there an easy way to query how many idle engines are
>> available?
>>
>> cheers,
>>
>> satra
>>
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20110119/a4ce4fe6/attachment.html>