spawning pyhon apps...

[Jason Scheirer]

On Jan 9, 4:07 pm, "bruce" <bedoug... at earthlink.net> wrote:
> thanks jason....
>
> or i could also, simply iterate through a loop of the names of the "child
> processes" i want to spawn, using the names in the subprocess.popen, and
> then proceeding with the rest of your example..
>
> question though... if i have a child app that's hanging.. how do i kill it.
>
> or is this getting into the aspect of using interprocess pipes, where if the
> parent isn't getting a 'live ping' via the pipe back from the child after a
> certain amount of time... it could kill the child...
>
> thoughts/comments...
>
> thanks
>
> -----Original Message-----
> From: python-list-bounces+bedouglas=earthlink.... at python.org
>
> [mailto:python-list-bounces+bedouglas=earthlink.... at python.org]On Behalf
> Of Jason Scheirer
> Sent: Friday, January 09, 2009 3:59 PM
> To: python-l... at python.org
> Subject: Re: spawning pyhon apps...
>
> On Jan 9, 3:43 pm, "bruce" <bedoug... at earthlink.net> wrote:
> > hi jason....
>
> > forgive me... but in your sample:
> >         my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
> >         %i.txt"%x) for x in xrange(10)]
> > are you spawning 'foo.py' 10 times? that can't be right!
> > so just what is "foo.py" used for? what am i missing...
>
> > it looks like the my_popenobjects array is iterated through to check the
> > statuscode. is the statuscode the value that would be returned from a
> child
> > python script via something like "return(2)"....
>
> > i've seen mention of os.waitpid(..) does this play into waiting for child
> > processes to complete, or determine if they've terminated??
>
> > thanks
>
> > -----Original Message-----
> > From: python-list-bounces+bedouglas=earthlink.... at python.org
>
> > [mailto:python-list-bounces+bedouglas=earthlink.... at python.org]On Behalf
> > Of Jason Scheirer
> > Sent: Friday, January 09, 2009 3:19 PM
> > To: python-l... at python.org
> > Subject: Re: spawning pyhon apps...
>
> > On Jan 9, 2:47 pm, "bruce" <bedoug... at earthlink.net> wrote:
> > > hi...
>
> > > toying with an idea.. trying to figure out a good/best way to spawn
> > multiple
> > > python scripts from a parent python app. i'm trying to figure out how to
> > > determine when all child apps have completed, or to possibly determine
> if
> > > any of the child processes have died/halted..
>
> > > parent app
> > >  spawn child1
> > >  spawn child2
> > >  spawn child3
> > >  .
> > >  .
> > >  .
> > >  spawn childn
>
> > > do i iterate through a os.waitpid(pid) for each pid of the child
> processes
> > i
> > > create?
>
> > > is there another approach? code samples/tutorial...??
>
> > > i've seen various approaches via google, but not just what i'm looking
> > for..
>
> > > thanks
>
> > Investigate the subprocess module, you probably want Popen objects.
> > You can do a poll loop like
>
> > my_popenobjects = [subprocess.Popen("foo.py", "--filename=file
> > %i.txt"%x) for x in xrange(10)]
>
> > while any(popenobject.returncode is None for popenobject in
> > my_popenobjects):
> >   time.sleep(0.25)
>
> > If your tasks are more like function calls and less like shell
> > scripts, then investigate writing your python as an importable module
> > and use the multiprocessing module, which will do threading/
> > subprocessing for you.
> > --http://mail.python.org/mailman/listinfo/python-list
>
> Correction: statuscode is wrong. It's returncode.
>
> Foo.py is the hypothetical Python script you want to run in a
> subprocess. In this example, I have an external shell script named
> foo.py, and I am indeed spawning 10 copies of it, each with a second
> argument that varies (foo.py --filename=file0.txt, foo.py --
> filename=file1.txt, ... foo.py --filename=file9.txt). You don't need
> os.waitpid() with a Popen object, there is a Popen.wait() method you
> can call which will accomplish the exact same thing. I'm polling the
> Popen.returncode for each process' return code (which is the numeric
> code a process returns when it finishes, like sys.exit(x) or return,
> or gives None if it's not done yet. What this sample is doing is
> opening 10 copies of the script and running them in parallel, if you
> want to run it is serial then you can do a simple for loop and .wait()
> on each:
>
> for cmd in ('a', 'b', 'c'):
>   sp = subprocess.Popen(cmd)
>   sp.wait()
>   print "Command %r completed with status %i" % (cmd, sp.returncode)
>
> I'm still not 100% sure what you're trying to accomplish. What is the
> exact problem you are wishing to solve?
> --http://mail.python.org/mailman/listinfo/python-list
>
>

Yes, so to open your processes you can loop over a list of commands
and create new subprocess.Popen(cmd) objects for each.

Everything is explained:

http://docs.python.org/library/subprocess.html#popen-objects

You can do .terminate() to kill a process that may be hanging, you can
get the .stdout and poll on its .read() to see if it's still putting
anything out to the console.