[IPython-dev] Using IPython Cluster with SGE -- help needed

[Matthieu Brucher]

Hi,

I guess we may want to start with the ipython documentation on this
topic: http://ipython.org/ipython-doc/stable/parallel/parallel_process.html

Cheers,

2013/8/4 Andreas Hilboll <lists at hilboll.de>:
> Hi,
>
> I would like to use IPython for calculations on our cluster. It's a
> total of 11 compute + 1 management nodes (all running Linux), and we're
> using SGE's qsub to submit jobs. The $HOME directory is shared via NFS
> between all the nodes.
>
> Even after reading the documentation, I'm unsure about how to get things
> running. I assume that I'll have to execute ``ìpcluster -n 16`` on all
> compute nodes (they have 16 cores each). I'd have the ipython shell
> (notebook won't work due to firewall restrictions I cannot change) on
> the management node. But how does the management node know about the
> kernels which are running on the compute nodes and waiting for a job?
> And how can I tell the management node that it shall use qsub to submit
> the jobs to the individual kernels?
>
> As I think this is a common use case, I'd be willing to write up a nice
> tutorial about the setup, but I fear I need some help from you guys to
> get things running ...
>
> Cheers,
>
> -- Andreas.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev



-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/