[Neuroimaging] Slow node starts with nipype

[Satrajit Ghosh]

hi jb,

it was because of people's usage patterns on clusters. however, we will
likely change the default back down in the upcoming release.

cheers,

satra

On Mon, Oct 31, 2016 at 12:07 PM, JB Poline <jbpoline at gmail.com> wrote:

> Hi Satra
>
> I'm just curious: What was the rationale for increasing this time from
> 2 to 30s ?
>
> cheers
> JB
>
> On 31 October 2016 at 09:02, Satrajit Ghosh <satra at mit.edu> wrote:
> > dear ian,
> >
> > there is a config option to sleep between nodes that was changed from 2s
> to
> > 60s. you can reset that option to speed things up.
> >
> > poll_sleep_duration
> >
> > http://nipype.readthedocs.io/en/latest/users/config_file.html
> >
> > you can set this to:
> >
> > wf_variable.config['execution']['poll_sleep_duraion'] = 2
> >
> > cheers,
> >
> > satra
> >
> > On Mon, Oct 31, 2016 at 11:54 AM, Ian Malone <ibmalone at gmail.com> wrote:
> >>
> >> Hi,
> >>
> >> I've been given some nipype workflows to run, and I find that they
> >> seem to go very slowly in between execution steps. For example, I'm
> >> currently looking at the message:
> >>
> >> 161031-15:42:50,278 workflow INFO:
> >>      Running: pm_scale -i
> >> /var/lib/midas/data/lha1946/images/nii/original/7938.nii.gz -o
> >>
> >> /var/drc/scratch1/malone/dti-test-masks-20161031/outgif/
> 13839424_01_PETMR_20161019-wd/dmri_workflow/susceptibility_
> correction_with_fm/pm_scale/7938_scaled.nii.gz
> >>
> >> But the command in question doesn't seem to be running (this is a
> >> fairly quick command if executed directly). There's a similar hold-up
> >> for other steps and, with quite a long multi-step pipeline, a lot of
> >> time seems to be spent doing nothing (watching top, similarly a list
> >> of fslmath nodes all launches, the fslmaths processes show up briefly
> >> in top, then there's a delay before anything more happens).
> >>
> >> Does anyone have suggestions for what might be happening? This is
> >> using the MultiProc engine, storage is NFS.
> >>
> >> --
> >> imalone
> >> _______________________________________________
> >> Neuroimaging mailing list
> >> Neuroimaging at python.org
> >> https://mail.python.org/mailman/listinfo/neuroimaging
> >
> >
> >
> > _______________________________________________
> > Neuroimaging mailing list
> > Neuroimaging at python.org
> > https://mail.python.org/mailman/listinfo/neuroimaging
> >
> _______________________________________________
> Neuroimaging mailing list
> Neuroimaging at python.org
> https://mail.python.org/mailman/listinfo/neuroimaging
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/neuroimaging/attachments/20161031/699576b7/attachment.html>