[IPython-dev] 0.11rc1 : problem with tutorial for PBS in http://ipython.org/ipython-doc/dev/parallel/parallel_process.html

[Johann Cohen-Tanugi]

hi there, my problem is in the fact that a line seems to be added to the 
template I am defining following the tutorial :
the template proposed in the tutorial is modified at runtime as :

#!/bin/sh
#PBS -t 1-4 <----------------- incorrect?
#PBS -V
#PBS -N ipengine
/usr/local/bin/python 
/sps/glast/users/cohen/IPYDEV/ipython/IPython/parallel/apps/ipengineapp.py 
profile_dir=/afs/in2p3.fr/home/t/tanugi/\
.ipython/profile_pbs


The problem I believe is in the job_array_template in  :

class PBSLauncher(BatchSystemLauncher):
     """A BatchSystemLauncher subclass for PBS."""

     submit_command = List(['qsub'], config=True,
         help="The PBS submit command ['qsub']")
     delete_command = List(['qdel'], config=True,
         help="The PBS delete command ['qsub']")
     job_id_regexp = Unicode(r'\d+', config=True,
         help="Regular expresion for identifying the job ID [r'\d+']")

     batch_file = Unicode(u'')
     job_array_regexp = Unicode('#PBS\W+-t\W+[\w\d\-\$]+')
     job_array_template = Unicode('#PBS -t 1-{n}')
     queue_regexp = Unicode('#PBS\W+-q\W+\$?\w+')
     queue_template = Unicode('#PBS -q {queue}')


I looked at the PBS doc for version 10 and 11 and I did not see any '-t' 
option. When I try to run, I get :
[tanugi at ccali28 test_directory]$ ipcluster start profile=pbs n=4
[IPClusterStart] Using existing profile dir: 
u'/afs/in2p3.fr/home/t/tanugi/.ipython/profile_pbs'
[IPClusterStart] Starting ipcluster with [daemon=False]
[IPClusterStart] Creating pid file: 
/afs/in2p3.fr/home/t/tanugi/.ipython/profile_pbs/pid/ipcluster.pid
[IPClusterStart] Starting PBSControllerLauncher: ['qsub', 
u'/afs/in2p3.fr/home/t/tanugi/.ipython/profile_pbs/pbs_controller']
[IPClusterStart] adding job array settings to batch script
[IPClusterStart] Writing instantiated batch script: 
/afs/in2p3.fr/home/t/tanugi/.ipython/profile_pbs/pbs_controller
unknown -t option
ERROR:root:Error in periodic callback
Traceback (most recent call last):
   File 
"/sps/glast/users/cohen/IPYDEV/local/lib/python2.6/site-packages/zmq/eventloop/ioloop.py", 
line 432, in _run
     self.callback()
   File 
"/sps/glast/users/cohen/IPYDEV/ipython/IPython/parallel/apps/ipclusterapp.py", 
line 364, in start_controller
     self.profile_dir.location
   File 
"/sps/glast/users/cohen/IPYDEV/ipython/IPython/parallel/apps/launcher.py", 
line 943, in start
     return super(PBSControllerLauncher, self).start(1, profile_dir)
   File 
"/sps/glast/users/cohen/IPYDEV/ipython/IPython/parallel/apps/launcher.py", 
line 902, in start
     job_id = self.parse_job_id(output)
   File 
"/sps/glast/users/cohen/IPYDEV/ipython/IPython/parallel/apps/launcher.py", 
line 854, in parse_job_id
     raise LauncherError("Job id couldn't be determined: %s" % output)
LauncherError: Job id couldn't be determined:

Not sure yet about the traceback, but the "unknown -t option" is clear. 
Furthermore, I wonder if it is really what we want to add lines to a 
template file provided by the user?

best,
Johann