[ python-Bugs-1048808 ] test_subprocess 2.4b1 fails on FreeBSD 5.2

[SourceForge.net]

Bugs item #1048808, was opened at 2004-10-17 21:24
Message generated for change (Comment added) made by astrand
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1048808&group_id=5470

Category: Python Library
Group: Python 2.4
Status: Open
Resolution: None
Priority: 7
Submitted By: Shalabh Chaturvedi (shalabh)
Assigned to: Peter Åstrand (astrand)
Summary: test_subprocess 2.4b1 fails on FreeBSD 5.2

Initial Comment:
Platform: FreeBSD 5.2.1-RELEASE
Python version: 2.4b1

After untarring I just did 'make' and then 'make test'.
make test passes all except following (output of make
test):
----
test_subprocess
    this bit of output is from a test of stdout in a
different process ...
test test_subprocess failed -- Traceback (most recent
call last):
  File
"/usr/home/shalabh/down/Python-2.4b1/Lib/test/test_subprocess.py",
line 432, in test_close_fds
    self.assertEqual(p.stdout.read(), "3")
AssertionError: '5' != '3'
----

Also, make test ends with:
----
*** Error code 1

Stop in /usr/home/shalabh/down/Python-2.4b1.
----
Don't know if above is expected when a test fails.


----------------------------------------------------------------------

>Comment By: Peter Åstrand (astrand)
Date: 2004-10-20 21:49

Message:
Logged In: YES 
user_id=344921

Strange. You could check if any other processes uses these
pipes with:

lsof | egrep "0xc541acc4|0xc541ad70"

----------------------------------------------------------------------

Comment By: Shalabh Chaturvedi (shalabh)
Date: 2004-10-20 20:38

Message:
Logged In: YES 
user_id=30293

They are pipes:

Python 2.4b1 (#1, Oct 17 2004, 12:04:08) 
[GCC 3.3.3 [FreeBSD] 20031106] on freebsd5
Type "help", "copyright", "credits" or "license" for more
information.
>>> import os, subprocess
>>> subprocess.call(["lsof", "-p", str(os.getpid())])
COMMAND  PID    USER   FD   TYPE     DEVICE SIZE/OFF    NODE
NAME
<snipped seemingly irrelevant .so files and folders>
python  1609 shalabh    0u  VCHR        5,1  0t15826     114
/dev/ttyp1
python  1609 shalabh    1u  VCHR        5,1  0t15826     114
/dev/ttyp1
python  1609 shalabh    2u  VCHR        5,1  0t15826     114
/dev/ttyp1
python  1609 shalabh    3u  PIPE 0xc541acc4    16384       
 ->0xc541ad70
python  1609 shalabh    4u  PIPE 0xc541ad70    16384       
 ->0xc541acc4
0


----------------------------------------------------------------------

Comment By: Peter Åstrand (astrand)
Date: 2004-10-20 20:26

Message:
Logged In: YES 
user_id=344921

Do you have lsof on your system? In that case, you could try
this:

>>> import os, subprocess
>>> subprocess.call(["lsof", "-p", str(os.getpid())])

That should give you a hint of what fd 3 and 4 really are. 

----------------------------------------------------------------------

Comment By: Shalabh Chaturvedi (shalabh)
Date: 2004-10-20 19:38

Message:
Logged In: YES 
user_id=30293

Same:

$ python -S
Python 2.3.4 (#2, Oct 15 2004, 13:44:35) 
[GCC 3.3.3 [FreeBSD] 20031106] on freebsd5
>>> import os                   
>>> os.dup(0)                   
5

The above behaviour is also seen with 2.4b1 using -S.

----------------------------------------------------------------------

Comment By: Fredrik Lundh (effbot)
Date: 2004-10-20 19:23

Message:
Logged In: YES 
user_id=38376

Just curious, but what happens if you start Python using the -
S option:

$ python -S
>>> import os
>>> os.dup(0)

----------------------------------------------------------------------

Comment By: Peter Åstrand (astrand)
Date: 2004-10-20 15:06

Message:
Logged In: YES 
user_id=344921

>>> import os
>>> os.dup(0)
5

I think this is the core of the problem. The test_close_fds
test works like this:

All file descriptors in the forked child (except 0,1,2) are
closed. Then the Python binary is executed via execvp(). A
small test program is passed to the Python binary via the -c
command line option. If the OS and subprocess module works
correctly, we can be sure of that by the time of the
execve() system call, only file descriptors (0,1,2) are open
(well, the errpipe as well, but let's leave that out for
now). But, by the time the Python binary starts executing
the small program, all sorts of things may have happened.
I'm not really sure we can trust Python not to open files
during startup. For example, if we have a PYTHONSTARTUP
file, that open file will have a file descriptor, perhaps 3. 

On one hand, this bug could indicate a bug in the Python
interpreter itself: perhaps a file descriptor leak. On the
other hand, this test might be a bit too unsafe. 

So probably, this test should be removed. 

----------------------------------------------------------------------

Comment By: Shalabh Chaturvedi (shalabh)
Date: 2004-10-17 23:06

Message:
Logged In: YES 
user_id=30293

Here's some more info. On my FreeBSD box, for both 2.3.4 and
2.4b1:
>>> import os
>>> os.dup(0)
5

But on my Win XP box for 2.3.3:
>>> import os
>>> os.dup(0)
3

I am guessing this may be relevant. Someone more
knowledgable can comment.



----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1048808&group_id=5470