[ python-Bugs-1460493 ] Why not drop the _active list?

[SourceForge.net]

Bugs item #1460493, was opened at 2006-03-29 09:16
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1460493&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
>Status: Closed
>Resolution: Fixed
Priority: 5
Submitted By: HVB bei TUP (hvb_tup)
Assigned to: Nobody/Anonymous (nobody)
Summary: Why not drop the _active list?

Initial Comment:
I am using a modified version of subprocess.py,

where I have removed the _active list and all 
references to it.

I have tested it (under Windows 2000) and there were 
no errors.

So what is the reason for managing the _active list 
at all? Why not drop it?

----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2006-04-10 17:58

Message:
Logged In: YES 
user_id=21627

This was fixed with said patch.

----------------------------------------------------------------------

Comment By: cheops (atila-cheops)
Date: 2006-04-10 16:55

Message:
Logged In: YES 
user_id=1276121

see patch #1467770

----------------------------------------------------------------------

Comment By: cheops (atila-cheops)
Date: 2006-04-05 10:27

Message:
Logged In: YES 
user_id=1276121

the implementation in 2.5 of popen2 seems good
if you or astrand could patch subprocess.py accordingly,
that would be great.

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-03-30 18:31

Message:
Logged In: YES 
user_id=21627

attila-cheops, please read the 2.5 version of popen2.

popen2 now only adds processes to _active in __del__, not in
__init__, so the problem with the application still wanting
to wait/poll is solved.

Multiple threads simultaneously isn't a problem, since it is
(or should be) harmless to invoke poll on a process that has
already been waited for. For only one of the poll calls, the
wait will actually succeed, and that should be the one that
removes it from the _active list.

The same strategy should be applied to subprocess.

----------------------------------------------------------------------

Comment By: cheops (atila-cheops)
Date: 2006-03-30 10:17

Message:
Logged In: YES 
user_id=1276121

also #1214859 is interesting, has a patch

----------------------------------------------------------------------

Comment By: cheops (atila-cheops)
Date: 2006-03-30 10:06

Message:
Logged In: YES 
user_id=1276121

the same problem probably exists in popen2.py
there _active is also used
so if something is fixed in subprocess.py, it should
probably also be fixed in popen2.py

----------------------------------------------------------------------

Comment By: cheops (atila-cheops)
Date: 2006-03-30 10:04

Message:
Logged In: YES 
user_id=1276121

what happens if you are doing a _cleanup (iterating over a
copy of _active) in multiple threads?
can it not happen then that you clean up a process 2 times?

thread 1 starts a _cleanup: makes a copy of _active[:] and
starts polling
thread 2 starts a _cleanup: makes a copy of _active[:] and
starts polling

thread 1 encounters a finished process and removes it from
_active[]
thread 2 does not know the process is removed, finds the
same process finished and tries to remove it from _active
but this fails, because thread 1 removed it already

so the action of cleaning up should maybe be serialized
if 1 thread is doing it, the other one should block

everyone who needs this can of course patch the
subprocess.py file, but shouldn't this be fixed in the library?

----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2006-03-30 09:43

Message:
Logged In: YES 
user_id=33168

If you always called wait() the _active list isn't
beneficial to you.  However, many people do not call wait
and the _active list provides a mechanism to cleanup zombied
children.  This is important for many users.

If you need thread safely, you can handle the locking
yourself before calling poll()/wait().

----------------------------------------------------------------------

Comment By: Martin v. Löwis (loewis)
Date: 2006-03-29 22:41

Message:
Logged In: YES 
user_id=21627

The purpose of the _active list is to wait(2) for open
processes. It needs to stay.

----------------------------------------------------------------------

Comment By: Tristan Faujour (tfaujour)
Date: 2006-03-29 15:59

Message:
Logged In: YES 
user_id=1488657

I agree.

The use of _active makes subprocess.py thread-UNsafe.

See also: Bug #1199282

In order to have a thread-safe subprocess.py, I commented
out the call to _cleanup() in Popen.__init__(). As a side
effect, _active becomes useless.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1460493&group_id=5470