[IPython-dev] SciPy Sprint summary

Sun Jul 18 14:24:41 EDT 2010

Hi,

I also prefer the first option, as it is the configuration I'm most
confortable with. Besides, people may have this already configured.

Matthieu

2010/7/18 Justin Riley <justin.t.riley at gmail.com>:
> Hi Matthieu,
>
> At least for the modifications I made, no not yet. This is exactly what I'm
> asking about in the second paragraph of my response. The new SGE/PBS support
> will work with multiple hosts assuming the ~/.ipython/security folder is
> NFS-shared on the cluster.
>
> If that's not the case, then AFAIK we have two options:
>
> 1. scp the furl file from ~/.ipython/security to each host's
> ~/.ipython/security folder.
>
> 2. put the contents of the furl file directly inside the job script
> used to start the engines
>
> The first option relies on the user having password-less configured properly
> to each node on the cluster. ipcluster would first need to scp the furl and
> then launch the engines using PBS/SGE.
>
> The second option is the easiest approach given that it only requires SGE to
> be installed, however, it's probably not the best idea to put the furl file
> in the job script itself for security reasons. I'm curious to get opinions
> on this. This would require slight code modifications.
>
> ~Justin
>
> On 07/18/2010 01:13 PM, Matthieu Brucher wrote:
>>
>> Hi,
>>
>> Does IPython support now sending engines to nodes that do not have the
>> same $HOME as the main instance? This is what kept me from testing
>> correctly IPython with LSF some months ago :|
>>
>> Matthieu
>>
>> 2010/7/18 Justin Riley<justin.t.riley at gmail.com>:
>>>
>>> Hi Satra/Brian,
>>>
>>> I modified your code to use the job array feature of SGE. I've also made
>>> it so that users don't need to specify --sge-script if they don't need a
>>> custom SGE launch script. My guess is that most users will choose not to
>>> specify --sge-script first and resort to using --sge-script when the
>>> generated launch script no longer meets their needs. More details in the
>>> git log here:
>>>
>>> http://github.com/jtriley/ipython/tree/0.10.1-sge
>>>
>>> Also, I need to test this, but I believe this code will fail if the
>>> folder containing the furl file is not NFS-mounted on the SGE cluster.
>>> Another option besides requiring NFS is to scp the furl file to each
>>> host as is done in the ssh mode of ipcluster, however, this would
>>> require password-less ssh to be configured properly (maybe not so bad).
>>> Another option is to dump the generated furl file into the job script
>>> itself. This has the advantage of only needing SGE installed but
>>> certainly doesn't seem like the safest practice. Any thoughts on how to
>>> approach this?
>>>
>>> Let me know what you think.
>>>
>>> ~Justin
>>>
>>> On 07/18/2010 12:05 AM, Brian Granger wrote:
>>>>
>>>> Is the array jobs feature what you want?
>>>>
>>>> http://wikis.sun.com/display/gridengine62u6/Submitting+Jobs
>>>>
>>>> Brian
>>>>
>>>> On Sat, Jul 17, 2010 at 9:00 PM, Brian Granger<ellisonbg at gmail.com>
>>>>  wrote:
>>>>>
>>>>> On Sat, Jul 17, 2010 at 6:23 AM, Satrajit Ghosh<satra at mit.edu>
>>>>>  wrote:
>>>>>>
>>>>>> hi ,
>>>>>>
>>>>>> i've pushed my changes to:
>>>>>>
>>>>>> http://github.com/satra/ipython/tree/0.10.1-sge
>>>>>>
>>>>>> notes:
>>>>>>
>>>>>> 1. it starts cleanly. i can connect and execute things. when i kill
>>>>>> using
>>>>>> ctrl-c, the messages appear to indicate that everything shut down
>>>>>> well.
>>>>>> however, the sge ipengine jobs are still running.
>>>>>
>>>>> What version of Python and Twisted are you running?
>>>>>
>>>>>> 2. the pbs option appears to require mpi to be present. i don't think
>>>>>> one
>>>>>> can launch multiple engines using pbs without mpi or without the
>>>>>> workaround
>>>>>> i've applied to the sge engine. basically it submits an sge job for
>>>>>> each
>>>>>> engine that i want to run. i would love to know if a single job can
>>>>>> launch
>>>>>> multiple engines on a sge/pbs cluster without mpi.
>>>>>
>>>>> I think you are right that pbs needs to use mpirun/mpiexec to start
>>>>> multiple engines using a single PBS job.  I am not that familiar with
>>>>> SGE, can you start mulitple processes without mpi and with just a
>>>>> single SGE job?  If so, let's try to get that working.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> Brian
>>>>>
>>>>>> cheers,
>>>>>>
>>>>>> satra
>>>>>>
>>>>>> On Thu, Jul 15, 2010 at 8:55 PM, Satrajit Ghosh<satra at mit.edu>
>>>>>>  wrote:
>>>>>>>
>>>>>>> hi justin,
>>>>>>>
>>>>>>> i hope to test it out tonight. from what fernando and i discussed,
>>>>>>> this
>>>>>>> should be relatively straightforward. once i'm done i'll push it to
>>>>>>> my fork
>>>>>>> of ipython and announce it here for others to test.
>>>>>>>
>>>>>>> cheers,
>>>>>>>
>>>>>>> satra
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jul 15, 2010 at 4:33 PM, Justin
>>>>>>> Riley<justin.t.riley at gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> This is great news. Right now StarCluster just takes advantage of
>>>>>>>> password-less ssh already being installed and runs:
>>>>>>>>
>>>>>>>> $ ipcluster ssh --clusterfile /path/to/cluster_file.py
>>>>>>>>
>>>>>>>> This works fine for now, however, having SGE support would allow
>>>>>>>> ipcluster's load to be accounted for by the queue.
>>>>>>>>
>>>>>>>> Is Satra on the list? I have experience with SGE and could help with
>>>>>>>> the
>>>>>>>> code if needed. I can also help test this functionality.
>>>>>>>>
>>>>>>>> ~Justin
>>>>>>>>
>>>>>>>> On 07/15/2010 03:34 PM, Fernando Perez wrote:
>>>>>>>>>
>>>>>>>>> On Thu, Jul 15, 2010 at 10:34 AM, Brian
>>>>>>>>> Granger<ellisonbg at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Thanks for the post.  You should also know that it looks like
>>>>>>>>>> someone
>>>>>>>>>> is going to add native SGE support to ipcluster for 0.10.1.
>>>>>>>>>
>>>>>>>>> Yes, Satra and I went over this last night in detail (thanks to
>>>>>>>>> Brian
>>>>>>>>> for the pointers), and he said he might actually already have some
>>>>>>>>> code for it.  I suspect we'll get this in soon.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> f
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> IPython-dev mailing list
>>>>>>>> IPython-dev at scipy.org
>>>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> IPython-dev mailing list
>>>>>> IPython-dev at scipy.org
>>>>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Brian E. Granger, Ph.D.
>>>>> Assistant Professor of Physics
>>>>> Cal Poly State University, San Luis Obispo
>>>>> bgranger at calpoly.edu
>>>>> ellisonbg at gmail.com
>>>>>
>>>>
>>>>
>>>>
>>>
>>> _______________________________________________
>>> IPython-dev mailing list
>>> IPython-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>>
>>
>>
>>
>
>

-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher