From noreply at sourceforge.net Fri Apr 1 00:13:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 00:13:34 2005 Subject: [ python-Bugs-1112549 ] cgi.FieldStorage memory usage can spike in line-oriented ops Message-ID: Bugs item #1112549, was opened at 2005-01-30 08:40 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris McDonough (chrism) Assigned to: Nobody/Anonymous (nobody) Summary: cgi.FieldStorage memory usage can spike in line-oriented ops Initial Comment: Various parts of cgi.FieldStorage call its "read_lines_to_outerboundary", "read_lines" and "skip_lines" methods. These methods use the "readline" method of the file object that represents an input stream. The input stream is typically data supplied by an untrusted source (such as a user uploading a file from a web browser). The input data is not required by the RFC 822/1521/1522/1867 specifications to contain any newline characters. For example, it is within the bounds of the specification to supply a a multipart/form-data input stream with a "file-data" part that consists of a 2GB string composed entirely of "x" characters (which happens to be something I did that led me to noticing this bug). The simplest fix is to make use of the "size" argument of the readline method of the file object where it is used within all parts of FieldStorage that make use of it. A patch against the Python 2.3.4 cgi.py module that does this is attached. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 17:13 Message: Logged In: YES user_id=6380 Methinks that the fix isn't quite right: it would incorrectly recognize as a boundary a very long line starting with "--" followed by the appropriate random string at offset 2**16. This could probably be taken care of by adding a flag that is true initially and after that keeps track of whether the previous line ended in \n. Also, there's a call to fp.readline() in parse_multipart() that you didn't patch -- it wouldn't help because that code is saving the lines in a list anyway, but isn't that code vulnerable as well? Or is it not used? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 From noreply at sourceforge.net Fri Apr 1 04:56:04 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 04:56:07 2005 Subject: [ python-Bugs-1112549 ] cgi.FieldStorage memory usage can spike in line-oriented ops Message-ID: Bugs item #1112549, was opened at 2005-01-30 08:40 Message generated for change (Comment added) made by chrism You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris McDonough (chrism) Assigned to: Nobody/Anonymous (nobody) Summary: cgi.FieldStorage memory usage can spike in line-oriented ops Initial Comment: Various parts of cgi.FieldStorage call its "read_lines_to_outerboundary", "read_lines" and "skip_lines" methods. These methods use the "readline" method of the file object that represents an input stream. The input stream is typically data supplied by an untrusted source (such as a user uploading a file from a web browser). The input data is not required by the RFC 822/1521/1522/1867 specifications to contain any newline characters. For example, it is within the bounds of the specification to supply a a multipart/form-data input stream with a "file-data" part that consists of a 2GB string composed entirely of "x" characters (which happens to be something I did that led me to noticing this bug). The simplest fix is to make use of the "size" argument of the readline method of the file object where it is used within all parts of FieldStorage that make use of it. A patch against the Python 2.3.4 cgi.py module that does this is attached. ---------------------------------------------------------------------- >Comment By: Chris McDonough (chrism) Date: 2005-03-31 21:56 Message: Logged In: YES user_id=32974 Re: parse_multipart.. yes, it looks like there's no use fixing that as it just turns around and puts the line into a list.. it is vulnerable but just by virtue of its non-use of a tempfile, it appears doomed anyway for large requests. I don't know of anything that uses it. Good catch wrt boundary recognition bug, I'm uploading another patch. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 17:13 Message: Logged In: YES user_id=6380 Methinks that the fix isn't quite right: it would incorrectly recognize as a boundary a very long line starting with "--" followed by the appropriate random string at offset 2**16. This could probably be taken care of by adding a flag that is true initially and after that keeps track of whether the previous line ended in \n. Also, there's a call to fp.readline() in parse_multipart() that you didn't patch -- it wouldn't help because that code is saving the lines in a list anyway, but isn't that code vulnerable as well? Or is it not used? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 From noreply at sourceforge.net Fri Apr 1 06:35:18 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 06:35:21 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 13:37 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-03-31 20:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 01:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-30 14:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Fri Apr 1 06:48:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 06:48:22 2005 Subject: [ python-Bugs-1112549 ] cgi.FieldStorage memory usage can spike in line-oriented ops Message-ID: Bugs item #1112549, was opened at 2005-01-30 08:40 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris McDonough (chrism) Assigned to: Nobody/Anonymous (nobody) Summary: cgi.FieldStorage memory usage can spike in line-oriented ops Initial Comment: Various parts of cgi.FieldStorage call its "read_lines_to_outerboundary", "read_lines" and "skip_lines" methods. These methods use the "readline" method of the file object that represents an input stream. The input stream is typically data supplied by an untrusted source (such as a user uploading a file from a web browser). The input data is not required by the RFC 822/1521/1522/1867 specifications to contain any newline characters. For example, it is within the bounds of the specification to supply a a multipart/form-data input stream with a "file-data" part that consists of a 2GB string composed entirely of "x" characters (which happens to be something I did that led me to noticing this bug). The simplest fix is to make use of the "size" argument of the readline method of the file object where it is used within all parts of FieldStorage that make use of it. A patch against the Python 2.3.4 cgi.py module that does this is attached. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 23:48 Message: Logged In: YES user_id=6380 Can I tweak you into uploading a unit test? ---------------------------------------------------------------------- Comment By: Chris McDonough (chrism) Date: 2005-03-31 21:56 Message: Logged In: YES user_id=32974 Re: parse_multipart.. yes, it looks like there's no use fixing that as it just turns around and puts the line into a list.. it is vulnerable but just by virtue of its non-use of a tempfile, it appears doomed anyway for large requests. I don't know of anything that uses it. Good catch wrt boundary recognition bug, I'm uploading another patch. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 17:13 Message: Logged In: YES user_id=6380 Methinks that the fix isn't quite right: it would incorrectly recognize as a boundary a very long line starting with "--" followed by the appropriate random string at offset 2**16. This could probably be taken care of by adding a flag that is true initially and after that keeps track of whether the previous line ended in \n. Also, there's a call to fp.readline() in parse_multipart() that you didn't patch -- it wouldn't help because that code is saving the lines in a list anyway, but isn't that code vulnerable as well? Or is it not used? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 From noreply at sourceforge.net Fri Apr 1 06:48:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 06:48:49 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 04:48 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Fri Apr 1 08:14:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 08:14:24 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 16:37 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 01:14 Message: Logged In: YES user_id=80475 'quit' and 'exit' currently show-up in a dir() of__builtins__. If the OP's suggestion is accepted, it should probably be implemented just like a builtin: def quit(): sys.exit() That approach is not at all magical and still allows shadowing (quit=2, etc.) What we have now is an annoying wart. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-03-31 23:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 04:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-30 17:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Fri Apr 1 11:22:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 11:22:17 2005 Subject: [ python-Bugs-1174712 ] subclassing ModuleType and another built-in type Message-ID: Bugs item #1174712, was opened at 2005-04-01 09:22 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174712&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Armin Rigo (arigo) Assigned to: Nobody/Anonymous (nobody) Summary: subclassing ModuleType and another built-in type Initial Comment: class X(types.ModuleType, str): pass X('name') -> segfault This buggy subclassing goes through typeobject.c's checks because PyModuleObject looks exactly like a user-defined subclass of 'object': it has a PyObject_HEAD followed only by the dict, as specified by tp_dictoffset. A fix would be to forbid any subclassing to move the tp_dictoffset of a non-heap type. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174712&group_id=5470 From noreply at sourceforge.net Fri Apr 1 11:58:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 11:58:57 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 04:48 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2005-04-01 09:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Fri Apr 1 13:04:43 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 13:04:47 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 23:37 Message generated for change (Comment added) made by sjoerd You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- >Comment By: Sjoerd Mullender (sjoerd) Date: 2005-04-01 13:04 Message: Logged In: YES user_id=43607 Something like this instead of the current quit should do the trick: class Quit: def __repr__(self): import sys sys.exit(0) quit = Quit() del Quit The problem with the Raymond's suggestion is that you need to type parentheses. But of course, this does get a little magical... ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 08:14 Message: Logged In: YES user_id=80475 'quit' and 'exit' currently show-up in a dir() of__builtins__. If the OP's suggestion is accepted, it should probably be implemented just like a builtin: def quit(): sys.exit() That approach is not at all magical and still allows shadowing (quit=2, etc.) What we have now is an annoying wart. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-01 06:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 11:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-31 00:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Fri Apr 1 13:12:12 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 13:12:15 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 21:37 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2005-04-01 11:12 Message: Logged In: YES user_id=4771 This discussion keeps coming up time and again. sjoerd's trick has been tried a few times already, until people implementing it realized that trying to display the builtins dict (directly or indirectly) would quit the interpreter! 'quit' cannot just be a synonym for 'sys.exit' either, because typing 'quit' would just print '', which is worse than the current situation in term of expliciteness. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2005-04-01 11:04 Message: Logged In: YES user_id=43607 Something like this instead of the current quit should do the trick: class Quit: def __repr__(self): import sys sys.exit(0) quit = Quit() del Quit The problem with the Raymond's suggestion is that you need to type parentheses. But of course, this does get a little magical... ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 06:14 Message: Logged In: YES user_id=80475 'quit' and 'exit' currently show-up in a dir() of__builtins__. If the OP's suggestion is accepted, it should probably be implemented just like a builtin: def quit(): sys.exit() That approach is not at all magical and still allows shadowing (quit=2, etc.) What we have now is an annoying wart. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-01 04:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 09:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-30 22:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Fri Apr 1 14:49:32 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 14:49:36 2005 Subject: [ python-Bugs-1174795 ] PYTHONPATH is not working Message-ID: Bugs item #1174795, was opened at 2005-04-01 15:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 Category: Windows Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Alexander Belchenko (bialix) Assigned to: Nobody/Anonymous (nobody) Summary: PYTHONPATH is not working Initial Comment: When I run python for interpreting script in form: > python somescript.py I got error: python: can't open file 'somescript.py': [Errno 2] No such file or directory I set environment variable PYTHONPATH to 'C:\Python24\Scripts'. In directory C:\Python24\Scripts I place script 'hello.py' with content: print 'Hello' This script simply prints 'Hello' word on the screen. When I run script with command: > python c:\python24\scripts\hello.py all works right and prints 'Hello'. When I run script with command 'python hello.py' I have above mentioned error: C:\>python hello.py python: can't open file 'hello.py': [Errno 2] No such file or directory I check that this directory appear in sys.path when invoking python in interpreter session: C:\>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', 'C:\Python24\Scripts', 'C:\Python24', 'C:\WINNT\system32\python24.zip ', 'C:\', 'C:\Python24\DLLs', 'C:\Python24\lib', 'C:\Python24\lib\plat-w in', 'C:\Python24\lib\lib-tk', 'C:\Python24\lib\site-packages', 'C:\Pytho n24\lib\site-packages\win32', 'C:\Python24\lib\site-packages\win32\lib', 'C:\Python24\lib\site-packages\Pythonwin'] >>> import hello Hello >>> As you see sys.path contain path to directory 'C:\Python24\Scripts' but nevertheless I cannot run my script hello.py simply type 'python hello.py' Changing of case of chars in string 'C:\Python24\Scripts' did not solved problem. This error has been reproduced on Windows 2000 SP4, Windows XP SP1, Windows 98 SE. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 From noreply at sourceforge.net Fri Apr 1 19:36:15 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 19:36:18 2005 Subject: [ python-Bugs-1174795 ] PYTHONPATH is not working Message-ID: Bugs item #1174795, was opened at 2005-04-01 21:49 Message generated for change (Comment added) made by quiver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 Category: Windows Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Alexander Belchenko (bialix) Assigned to: Nobody/Anonymous (nobody) Summary: PYTHONPATH is not working Initial Comment: When I run python for interpreting script in form: > python somescript.py I got error: python: can't open file 'somescript.py': [Errno 2] No such file or directory I set environment variable PYTHONPATH to 'C:\Python24\Scripts'. In directory C:\Python24\Scripts I place script 'hello.py' with content: print 'Hello' This script simply prints 'Hello' word on the screen. When I run script with command: > python c:\python24\scripts\hello.py all works right and prints 'Hello'. When I run script with command 'python hello.py' I have above mentioned error: C:\>python hello.py python: can't open file 'hello.py': [Errno 2] No such file or directory I check that this directory appear in sys.path when invoking python in interpreter session: C:\>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', 'C:\Python24\Scripts', 'C:\Python24', 'C:\WINNT\system32\python24.zip ', 'C:\', 'C:\Python24\DLLs', 'C:\Python24\lib', 'C:\Python24\lib\plat-w in', 'C:\Python24\lib\lib-tk', 'C:\Python24\lib\site-packages', 'C:\Pytho n24\lib\site-packages\win32', 'C:\Python24\lib\site-packages\win32\lib', 'C:\Python24\lib\site-packages\Pythonwin'] >>> import hello Hello >>> As you see sys.path contain path to directory 'C:\Python24\Scripts' but nevertheless I cannot run my script hello.py simply type 'python hello.py' Changing of case of chars in string 'C:\Python24\Scripts' did not solved problem. This error has been reproduced on Windows 2000 SP4, Windows XP SP1, Windows 98 SE. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2005-04-02 02:36 Message: Logged In: YES user_id=671362 PYTHONPATH is a path to search for modules, not a path to search for executable programs like PATH. If PATH variable contains a path to where hello.py is located, c:\> hello.py should work. Please read the tutorial about PYTHONPATH. http://docs.python.org/tut/node8.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 From noreply at sourceforge.net Fri Apr 1 20:13:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 20:13:54 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 16:37 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 13:13 Message: Logged In: YES user_id=80475 class quit: def __repr__(self): return "Type quit() to exit the interpreter" def __call__(self): sys.exit() Requiring the parentheses is not a big deal as long as there is an advisory where they are omitted. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 06:12 Message: Logged In: YES user_id=4771 This discussion keeps coming up time and again. sjoerd's trick has been tried a few times already, until people implementing it realized that trying to display the builtins dict (directly or indirectly) would quit the interpreter! 'quit' cannot just be a synonym for 'sys.exit' either, because typing 'quit' would just print '', which is worse than the current situation in term of expliciteness. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2005-04-01 06:04 Message: Logged In: YES user_id=43607 Something like this instead of the current quit should do the trick: class Quit: def __repr__(self): import sys sys.exit(0) quit = Quit() del Quit The problem with the Raymond's suggestion is that you need to type parentheses. But of course, this does get a little magical... ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 01:14 Message: Logged In: YES user_id=80475 'quit' and 'exit' currently show-up in a dir() of__builtins__. If the OP's suggestion is accepted, it should probably be implemented just like a builtin: def quit(): sys.exit() That approach is not at all magical and still allows shadowing (quit=2, etc.) What we have now is an annoying wart. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-03-31 23:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 04:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-30 17:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Fri Apr 1 20:35:30 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 20:35:41 2005 Subject: [ python-Bugs-997100 ] optparse.py:668 triggers FutureWarning Message-ID: Bugs item #997100, was opened at 2004-07-24 10:20 Message generated for change (Comment added) made by hydrian You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=997100&group_id=5470 Category: Python Library Group: Python 2.3 Status: Closed Resolution: Fixed Priority: 3 Submitted By: Charles (melicertes) Assigned to: Greg Ward (gward) Summary: optparse.py:668 triggers FutureWarning Initial Comment: In Python 2.3.3, importing optparse triggers a FutureWarning on line 668: > /usr/local/lib/python2.3/optparse.py:668: FutureWarning: > %u/%o/%x/%X of negative int will return a signed string in > Python 2.4 and up > return ("<%s at 0x%x: %r>" ---------------------------------------------------------------------- Comment By: Ben Tyger (hydrian) Date: 2005-04-01 13:35 Message: Logged In: YES user_id=685946 I am still seeing issues with Python 2.3.5 with getmail 4.3.3 ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2004-11-05 21:08 Message: Logged In: YES user_id=14422 Closing since (I think) I fixed this in Optik 1.5a2. ---------------------------------------------------------------------- Comment By: Matthias Andree (m-a) Date: 2004-09-25 06:27 Message: Logged In: YES user_id=2788 I for one have seen the warning with a Python 2.3.4 I compiled myself (based on SuSE's 2.3.3 source RPM) on SuSE Linux 9.1. ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2004-09-24 20:58 Message: Logged In: YES user_id=14422 Marking this fixed, but not closing it, pending a checkin to optparse.py on Python 2.3 branch and confirmation that it actually helped. ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2004-09-24 20:58 Message: Logged In: YES user_id=14422 You didn't specify which OS this happens on; I bet it was Fedora Core or RHEL 3.0, right? (I found a related thread on python-dev, and ISTR it was related to the SE-Linux changes fiddling how the userspace heap fits into VM.) I'm checking in a change to optik/lib/option.py that should suppress the warning. I'm not sure this is the right thing to do, though, since 1) I can't reproduce it, and 2) I don't think it will be an issue under Python 2.4, with the continuing unification of int and long. See http://mail.python.org/pipermail/python-dev/2003-November/039952.html and following messages. Perhaps the right thing to do is just fix this in optparse.py on the Python 2.3 branch and forget about Optik or Python 2.4. Hmmm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=997100&group_id=5470 From noreply at sourceforge.net Fri Apr 1 22:09:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 22:09:38 2005 Subject: [ python-Bugs-1175022 ] property example code error Message-ID: Bugs item #1175022, was opened at 2005-04-01 20:09 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175022&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: John Ridley (ojokimu) Assigned to: Nobody/Anonymous (nobody) Summary: property example code error Initial Comment: The example code for 'property' in lib/built-in-funcs.html may produce an error if run "as is": Python 2.4.1 (#1, Mar 31 2005, 21:33:58) [GCC 3.4.1 (Mandrakelinux (Alpha 3.4.1-3mdk)] on linux2 >>> class C(object): ... def getx(self): return self.__x ... def setx(self, value): self.__x = value ... def delx(self): del self.__x ... x = property(getx, setx, delx, "I'm the 'x' property.") ... >>> c=C() >>> c.x Traceback (most recent call last): File "", line 1, in ? File "", line 2, in getx AttributeError: 'C' object has no attribute '_C__x' The same goes for 'del c.x' (although not 'c.x = 0', of course). A more "typical" way of defining managed attributes would be to include an 'init' as follows: class C(object): def __init__(self): self.__x = None def getx(self): return self.__x def setx(self, value): self.__x = value def delx(self): del self.__x x = property(getx, setx, delx, "I'm the 'x' property.") ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175022&group_id=5470 From noreply at sourceforge.net Fri Apr 1 23:02:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 23:02:05 2005 Subject: [ python-Bugs-1174795 ] PYTHONPATH is not working Message-ID: Bugs item #1174795, was opened at 2005-04-01 04:49 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 Category: Windows Group: Python 2.4 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Alexander Belchenko (bialix) Assigned to: Nobody/Anonymous (nobody) Summary: PYTHONPATH is not working Initial Comment: When I run python for interpreting script in form: > python somescript.py I got error: python: can't open file 'somescript.py': [Errno 2] No such file or directory I set environment variable PYTHONPATH to 'C:\Python24\Scripts'. In directory C:\Python24\Scripts I place script 'hello.py' with content: print 'Hello' This script simply prints 'Hello' word on the screen. When I run script with command: > python c:\python24\scripts\hello.py all works right and prints 'Hello'. When I run script with command 'python hello.py' I have above mentioned error: C:\>python hello.py python: can't open file 'hello.py': [Errno 2] No such file or directory I check that this directory appear in sys.path when invoking python in interpreter session: C:\>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', 'C:\Python24\Scripts', 'C:\Python24', 'C:\WINNT\system32\python24.zip ', 'C:\', 'C:\Python24\DLLs', 'C:\Python24\lib', 'C:\Python24\lib\plat-w in', 'C:\Python24\lib\lib-tk', 'C:\Python24\lib\site-packages', 'C:\Pytho n24\lib\site-packages\win32', 'C:\Python24\lib\site-packages\win32\lib', 'C:\Python24\lib\site-packages\Pythonwin'] >>> import hello Hello >>> As you see sys.path contain path to directory 'C:\Python24\Scripts' but nevertheless I cannot run my script hello.py simply type 'python hello.py' Changing of case of chars in string 'C:\Python24\Scripts' did not solved problem. This error has been reproduced on Windows 2000 SP4, Windows XP SP1, Windows 98 SE. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-01 13:02 Message: Logged In: YES user_id=357491 George is right. Closing as invalid. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2005-04-01 09:36 Message: Logged In: YES user_id=671362 PYTHONPATH is a path to search for modules, not a path to search for executable programs like PATH. If PATH variable contains a path to where hello.py is located, c:\> hello.py should work. Please read the tutorial about PYTHONPATH. http://docs.python.org/tut/node8.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 From noreply at sourceforge.net Fri Apr 1 23:42:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 23:42:50 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 06:48 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 23:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 11:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Fri Apr 1 23:47:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 23:47:27 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 23:37 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 23:47 Message: Logged In: YES user_id=21627 But how is this better than the current >>> quit 'Use Ctrl-D (i.e. EOF) to exit.' I'd rather learn that I have to type Ctrl-D instead of typing quit() I think the original report is misguided: The interpreter does not "obviously understand" the quit command. Instead, it really does not understand it all. The name quit is bound to a string, and the interpreter displays the string. It does not understand at all what the string says. So I'm rejecting the report as "won't fix". ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 20:13 Message: Logged In: YES user_id=80475 class quit: def __repr__(self): return "Type quit() to exit the interpreter" def __call__(self): sys.exit() Requiring the parentheses is not a big deal as long as there is an advisory where they are omitted. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 13:12 Message: Logged In: YES user_id=4771 This discussion keeps coming up time and again. sjoerd's trick has been tried a few times already, until people implementing it realized that trying to display the builtins dict (directly or indirectly) would quit the interpreter! 'quit' cannot just be a synonym for 'sys.exit' either, because typing 'quit' would just print '', which is worse than the current situation in term of expliciteness. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2005-04-01 13:04 Message: Logged In: YES user_id=43607 Something like this instead of the current quit should do the trick: class Quit: def __repr__(self): import sys sys.exit(0) quit = Quit() del Quit The problem with the Raymond's suggestion is that you need to type parentheses. But of course, this does get a little magical... ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 08:14 Message: Logged In: YES user_id=80475 'quit' and 'exit' currently show-up in a dir() of__builtins__. If the OP's suggestion is accepted, it should probably be implemented just like a builtin: def quit(): sys.exit() That approach is not at all magical and still allows shadowing (quit=2, etc.) What we have now is an annoying wart. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-01 06:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 11:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-31 00:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Fri Apr 1 23:50:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 23:51:01 2005 Subject: [ python-Bugs-1170424 ] why should URL be required for all packages Message-ID: Bugs item #1170424, was opened at 2005-03-25 10:58 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1170424&group_id=5470 Category: Distutils Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: J?rgen A. Erhard (jae) Assigned to: Nobody/Anonymous (nobody) Summary: why should URL be required for all packages Initial Comment: Annoying if you just want to roll an in-house package. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 23:50 Message: Logged In: YES user_id=21627 In addition, the bug report is under-specified: Who says that an URL is required for all packages? It certainly is possible to have packages without URL, so I can't see why you are annoyed. Closing the report as "invalid". If you still think there is a bug in Python, please submit a new report, stating what you are doing, what happens, and what you expect to happen instead. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-03-30 20:12 Message: Logged In: YES user_id=593130 Bugs are a discrepancy between intended/claimed behavior and actual behavior. Since this does not appear to report such a discrepancy, it does not appear to be a bug report. Please close and ask your question and make your comment on the Python discussion mailing list or newsgroup (comp.lang.python). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1170424&group_id=5470 From noreply at sourceforge.net Fri Apr 1 23:55:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 1 23:55:44 2005 Subject: [ python-Bugs-1169633 ] Install fail code 2932 after fail to copy python_icon.exe Message-ID: Bugs item #1169633, was opened at 2005-03-24 05:04 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1169633&group_id=5470 Category: Installation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Wagner (wks2) Assigned to: Nobody/Anonymous (nobody) Summary: Install fail code 2932 after fail to copy python_icon.exe Initial Comment: In the Enterprise Windows environment we have a Network share as the homedir for all users. The server forbid to write .exe files to this share. I can install programs in the local disk (I have lots of then installed). Python-2.4.MSI Installation fails with error code 2932 after the system rejects the copy of python_icon.exe to the homedir. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 23:55 Message: Logged In: YES user_id=21627 Please provide a little more details on how to reproduce this: What operating system are you using? How precisely is the server setup to forbid writing .exe files to a share? Are you trying to install the program as a machine administrator, or as a non-administrator? If administrator, did you select per-user or per-machine installations? As is, it is very hard to understand what you did. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1169633&group_id=5470 From noreply at sourceforge.net Sat Apr 2 00:02:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 00:02:23 2005 Subject: [ python-Bugs-1167262 ] Fails assertion in winsig.c under VC 8.0 Message-ID: Bugs item #1167262, was opened at 2005-03-21 04:25 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167262&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Timothy Fitz (timothyfitz) Assigned to: Nobody/Anonymous (nobody) Summary: Fails assertion in winsig.c under VC 8.0 Initial Comment: In 2.4 and current cvs initsignal in signalmodule.c calls PyOS_getsig which calls signal with an invalid signal number. Under VC 7.1 (and 7.0, and probably before) this would return SIG_ERR however under VC 8.0 (beta) this causes an assertion failure. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-02 00:02 Message: Logged In: YES user_id=21627 C99 7.14.1.1/p8 specifies # If the request can be honored, the signal function # returns the value of func for the most recent successful # call to signal for the specified signal sig. Otherwise, a # value of SIG_ERR is returned and a positive value is stored # in errno. So it seems to me that VC8 is in error: the request cannot be "honored", so it should return SIG_ERR. Until/if VC8 is released with that bug, I would like to take no action here. When it is released, we still might consider to ignore it until say, 8.1 comes along and fixes the bug. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-27 22:59 Message: Logged In: YES user_id=6656 Ew. My first thought is, is that allowed by ANSI C? I'd guess it's undefined behaviour. It still seems excessively unfriendly -- have you reported this to MS? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1167262&group_id=5470 From noreply at sourceforge.net Sat Apr 2 05:00:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 05:00:49 2005 Subject: [ python-Bugs-997100 ] optparse.py:668 triggers FutureWarning Message-ID: Bugs item #997100, was opened at 2004-07-24 11:20 Message generated for change (Comment added) made by gward You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=997100&group_id=5470 Category: Python Library Group: Python 2.3 Status: Closed Resolution: Fixed Priority: 3 Submitted By: Charles (melicertes) Assigned to: Greg Ward (gward) Summary: optparse.py:668 triggers FutureWarning Initial Comment: In Python 2.3.3, importing optparse triggers a FutureWarning on line 668: > /usr/local/lib/python2.3/optparse.py:668: FutureWarning: > %u/%o/%x/%X of negative int will return a signed string in > Python 2.4 and up > return ("<%s at 0x%x: %r>" ---------------------------------------------------------------------- >Comment By: Greg Ward (gward) Date: 2005-04-01 22:00 Message: Logged In: YES user_id=14422 It's highly unlikely there will be another Python 2.3 release. If you still see this behaviour with 2.4 or 2.4.1, let me know and I'll consider fixing it in 2.4. Otherwise, this bug stayss closed. Sorry. ---------------------------------------------------------------------- Comment By: Ben Tyger (hydrian) Date: 2005-04-01 13:35 Message: Logged In: YES user_id=685946 I am still seeing issues with Python 2.3.5 with getmail 4.3.3 ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2004-11-05 21:08 Message: Logged In: YES user_id=14422 Closing since (I think) I fixed this in Optik 1.5a2. ---------------------------------------------------------------------- Comment By: Matthias Andree (m-a) Date: 2004-09-25 07:27 Message: Logged In: YES user_id=2788 I for one have seen the warning with a Python 2.3.4 I compiled myself (based on SuSE's 2.3.3 source RPM) on SuSE Linux 9.1. ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2004-09-24 21:58 Message: Logged In: YES user_id=14422 Marking this fixed, but not closing it, pending a checkin to optparse.py on Python 2.3 branch and confirmation that it actually helped. ---------------------------------------------------------------------- Comment By: Greg Ward (gward) Date: 2004-09-24 21:58 Message: Logged In: YES user_id=14422 You didn't specify which OS this happens on; I bet it was Fedora Core or RHEL 3.0, right? (I found a related thread on python-dev, and ISTR it was related to the SE-Linux changes fiddling how the userspace heap fits into VM.) I'm checking in a change to optik/lib/option.py that should suppress the warning. I'm not sure this is the right thing to do, though, since 1) I can't reproduce it, and 2) I don't think it will be an issue under Python 2.4, with the continuing unification of int and long. See http://mail.python.org/pipermail/python-dev/2003-November/039952.html and following messages. Perhaps the right thing to do is just fix this in optparse.py on the Python 2.3 branch and forget about Optik or Python 2.4. Hmmm. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=997100&group_id=5470 From noreply at sourceforge.net Sat Apr 2 06:46:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 06:47:00 2005 Subject: [ python-Bugs-1174795 ] PYTHONPATH is not working Message-ID: Bugs item #1174795, was opened at 2005-04-01 15:49 Message generated for change (Comment added) made by bialix You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 Category: Windows Group: Python 2.4 Status: Closed Resolution: Invalid Priority: 5 Submitted By: Alexander Belchenko (bialix) Assigned to: Nobody/Anonymous (nobody) Summary: PYTHONPATH is not working Initial Comment: When I run python for interpreting script in form: > python somescript.py I got error: python: can't open file 'somescript.py': [Errno 2] No such file or directory I set environment variable PYTHONPATH to 'C:\Python24\Scripts'. In directory C:\Python24\Scripts I place script 'hello.py' with content: print 'Hello' This script simply prints 'Hello' word on the screen. When I run script with command: > python c:\python24\scripts\hello.py all works right and prints 'Hello'. When I run script with command 'python hello.py' I have above mentioned error: C:\>python hello.py python: can't open file 'hello.py': [Errno 2] No such file or directory I check that this directory appear in sys.path when invoking python in interpreter session: C:\>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.path ['', 'C:\Python24\Scripts', 'C:\Python24', 'C:\WINNT\system32\python24.zip ', 'C:\', 'C:\Python24\DLLs', 'C:\Python24\lib', 'C:\Python24\lib\plat-w in', 'C:\Python24\lib\lib-tk', 'C:\Python24\lib\site-packages', 'C:\Pytho n24\lib\site-packages\win32', 'C:\Python24\lib\site-packages\win32\lib', 'C:\Python24\lib\site-packages\Pythonwin'] >>> import hello Hello >>> As you see sys.path contain path to directory 'C:\Python24\Scripts' but nevertheless I cannot run my script hello.py simply type 'python hello.py' Changing of case of chars in string 'C:\Python24\Scripts' did not solved problem. This error has been reproduced on Windows 2000 SP4, Windows XP SP1, Windows 98 SE. ---------------------------------------------------------------------- >Comment By: Alexander Belchenko (bialix) Date: 2005-04-02 07:46 Message: Logged In: YES user_id=957594 Sorry for this mistake. Obviously I misunderstood documentation. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-02 00:02 Message: Logged In: YES user_id=357491 George is right. Closing as invalid. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2005-04-01 20:36 Message: Logged In: YES user_id=671362 PYTHONPATH is a path to search for modules, not a path to search for executable programs like PATH. If PATH variable contains a path to where hello.py is located, c:\> hello.py should work. Please read the tutorial about PYTHONPATH. http://docs.python.org/tut/node8.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174795&group_id=5470 From noreply at sourceforge.net Sat Apr 2 07:11:10 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 07:11:13 2005 Subject: [ python-Bugs-1175194 ] import statement likely to crash if module launches threads Message-ID: Bugs item #1175194, was opened at 2005-04-01 21:11 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175194&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Jeff Stearns (jeffstearns) Assigned to: Nobody/Anonymous (nobody) Summary: import statement likely to crash if module launches threads Initial Comment: I have a simple module which launches multiple HTTP client threads. The main thread creates 10 HTTP clients, each of which fetches documents from a web server. The main thread then simply goes to sleep while the client threads work. The threads are launched when the module is imported. If I launch the script via python bug1.py it launches and runs OK, and dies cleanly upon ^C. But if I launch the script via python -c 'import bug1' it hangs at launch, and dumps core upon ^C. Here's an example: jps@cure> ./python -c 'import bug1' Using 10 threads cccccccccc <- [program hangs here] Traceback (most recent call last): File "", line 1, in ? File "test/bug1.py", line 55, in ? run (10) File "test/bug1.py", line 50, in run time.sleep (30000) KeyboardInterrupt Fatal Python error: PyImport_GetModuleDict: no module dictionary! Aborted (core dumped) You might argue that it's inappropriate to launch threads from within import statement, but I can't find a specific prohibition against it. In any event, it shouldn't cause the interpreter to crash. Please note that the crash isn't consistent. Sometimes the import statement doesn't lead to a crash. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175194&group_id=5470 From noreply at sourceforge.net Sat Apr 2 07:24:51 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 07:24:54 2005 Subject: [ python-Bugs-1175202 ] python hangs if import statement launches threads Message-ID: Bugs item #1175202, was opened at 2005-04-01 21:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175202&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Jeff Stearns (jeffstearns) Assigned to: Nobody/Anonymous (nobody) Summary: python hangs if import statement launches threads Initial Comment: (This bug looks similar to bug 1175194, but reports a different problem.) I have a simple module which launches multiple HTTP client threads. The main thread creates 10 HTTP clients, each of which fetches documents from a web server. The main thread then simply goes to sleep while the client threads work. The threads are launched when the module is imported. If I launch the script via python bug1.py it launches and runs OK. But if I launch the script via python -c 'import bug1' it runs a bit, then hangs. Here's an example: jps@cure> ./python -c 'import bug1' Using 10 threads cccccccccc <- [program hangs here] Each thread prints a character every time that it does something interesting. The 'c' characters indicate that a connect syscall was made. These should be followed by 'C', indicating the the connect returned. That never happens. You might argue that it's inappropriate to launch threads from within import statement, but I can't find a specific prohibition against it. Here's a trace of the last few syscalls before it hangs (pids are actually thread ids): [pid 15481] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15481] futex(0x8148698, FUTEX_WAIT, 0, NULL [pid 15482] futex(0x81a9a80, FUTEX_WAKE, 1) = 0 [pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15482] write(2, "c", 1c) = 1 [pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15482] futex(0x8148698, FUTEX_WAIT, 0, NULL <- hangs here Please note that this bug is similar to bug 1175194, but reports a different problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175202&group_id=5470 From noreply at sourceforge.net Sat Apr 2 14:24:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 14:24:23 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 21:37 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2005-04-02 12:24 Message: Logged In: YES user_id=4771 Raymond, an argument against quit() actually quitting the interpreter is that it suddenly makes quit() a quasi-official piece of the Python API, and we're bound to see user programs start to write quit() instead of sys.exit(). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 21:47 Message: Logged In: YES user_id=21627 But how is this better than the current >>> quit 'Use Ctrl-D (i.e. EOF) to exit.' I'd rather learn that I have to type Ctrl-D instead of typing quit() I think the original report is misguided: The interpreter does not "obviously understand" the quit command. Instead, it really does not understand it all. The name quit is bound to a string, and the interpreter displays the string. It does not understand at all what the string says. So I'm rejecting the report as "won't fix". ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 18:13 Message: Logged In: YES user_id=80475 class quit: def __repr__(self): return "Type quit() to exit the interpreter" def __call__(self): sys.exit() Requiring the parentheses is not a big deal as long as there is an advisory where they are omitted. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 11:12 Message: Logged In: YES user_id=4771 This discussion keeps coming up time and again. sjoerd's trick has been tried a few times already, until people implementing it realized that trying to display the builtins dict (directly or indirectly) would quit the interpreter! 'quit' cannot just be a synonym for 'sys.exit' either, because typing 'quit' would just print '', which is worse than the current situation in term of expliciteness. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2005-04-01 11:04 Message: Logged In: YES user_id=43607 Something like this instead of the current quit should do the trick: class Quit: def __repr__(self): import sys sys.exit(0) quit = Quit() del Quit The problem with the Raymond's suggestion is that you need to type parentheses. But of course, this does get a little magical... ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 06:14 Message: Logged In: YES user_id=80475 'quit' and 'exit' currently show-up in a dir() of__builtins__. If the OP's suggestion is accepted, it should probably be implemented just like a builtin: def quit(): sys.exit() That approach is not at all magical and still allows shadowing (quit=2, etc.) What we have now is an annoying wart. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-01 04:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 09:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-30 22:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Sat Apr 2 14:27:54 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 14:27:57 2005 Subject: [ python-Bugs-1174712 ] subclassing ModuleType and another built-in type Message-ID: Bugs item #1174712, was opened at 2005-04-01 09:22 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174712&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Armin Rigo (arigo) Assigned to: Nobody/Anonymous (nobody) Summary: subclassing ModuleType and another built-in type Initial Comment: class X(types.ModuleType, str): pass X('name') -> segfault This buggy subclassing goes through typeobject.c's checks because PyModuleObject looks exactly like a user-defined subclass of 'object': it has a PyObject_HEAD followed only by the dict, as specified by tp_dictoffset. A fix would be to forbid any subclassing to move the tp_dictoffset of a non-heap type. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2005-04-02 12:27 Message: Logged In: YES user_id=4771 The proposed fix is not good enough. If another built-in C type similar to PyModuleObject is defined (say by a C extension module), then it would become possible to subclass both this new type and ModuleType. This might lead to unwanted behavior if e.g. one parent class allows rebinding __dict__ and the other not (and crashes if __dict__ is rebound). This might point to the need for cleaning up typeobject.c's darker corners... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174712&group_id=5470 From noreply at sourceforge.net Sat Apr 2 14:32:00 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 14:32:02 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 04:48 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2005-04-02 12:31 Message: Logged In: YES user_id=4771 os.stat() doesn't always give consistent results on dev files. On my machine for some reason os.stat('/dev/null') appears to be random (and extremely large). I suspect that on the OP's machine os.stat('/dev/zero') is not 0 either, but a random number that turns out to be negative, hence a "bad argument" SystemError. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 21:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 09:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Sat Apr 2 17:14:44 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 17:14:46 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 17:14 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Sat Apr 2 23:44:55 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 2 23:44:59 2005 Subject: [ python-Bugs-1163401 ] uncaught AttributeError deep in urllib Message-ID: Bugs item #1163401, was opened at 2005-03-14 19:39 Message generated for change (Comment added) made by roysmith You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163401&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: K Lars Lohn (lohnk) Assigned to: Nobody/Anonymous (nobody) Summary: uncaught AttributeError deep in urllib Initial Comment: Python 2.4 and Python 2.3.4 running under Suse 9.2 We're getting an AttributeError exception "AttributeError: 'NoneType' object has no attribute 'read'" within a very simple call to urllib.urlopen. This was discovered while working on Sentry 2, the new mirror integrity checker for the Mozilla project. We try to touch hundreds of URLs to make sure that the files are present on each of the mirrors. One particular URL kills the call to urllib.urlopen: http://mozilla.mirrors.skynet.be/pub/ftp.mozilla.org/firefox/releases/1.0/win32/en-US/Firefox%20Setup%201.0.exe This file probably does not exist on the mirror, however, in other cases of bad URLs, we get much more graceful failures when we try to read from the object returned by urllib.urlopen. >>> import urllib >>> urlReader = urllib.urlopen("http://mozilla.mirrors.skynet.be/pub/ftp.mozilla.org/firefox/releases/1.0/win32/en-US/Firefox%20Setup%201.0.exe") Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.4/urllib.py", line 77, in urlopen return opener.open(url) File "/usr/local/lib/python2.4/urllib.py", line 180, in open return getattr(self, name)(url) File "/usr/local/lib/python2.4/urllib.py", line 305, in open_http return self.http_error(url, fp, errcode, errmsg, headers) File "/usr/local/lib/python2.4/urllib.py", line 322, in http_error return self.http_error_default(url, fp, errcode, errmsg, headers) File "/usr/local/lib/python2.4/urllib.py", line 550, in http_error_default return addinfourl(fp, headers, "http:" + url) File "/usr/local/lib/python2.4/urllib.py", line 836, in __init__ addbase.__init__(self, fp) File "/usr/local/lib/python2.4/urllib.py", line 786, in __init__ self.read = self.fp.read AttributeError: 'NoneType' object has no attribute 'read' The attached file is a three line scipt that demos the problem. ---------------------------------------------------------------------- Comment By: Roy Smith (roysmith) Date: 2005-04-02 16:44 Message: Logged In: YES user_id=390499 Wow, this is bizarre. I just spend some time tracking down exactly this same bug and was just about to enter it when I saw this entry. For what it's worth, I can reliably reproduce this exception when fetching a URL from a deliberately broken server (well, at least I think it's broken; have to double-check the HTTP spec to be sure this isn't actually allowed) which produces headers but no body: (This is on Mac OSX-10.3.8, Python-2.3.4) ------------------------------- Roy-Smiths-Computer:bug$ cat server.py #!/usr/bin/env python from BaseHTTPServer import * class NullHandler (BaseHTTPRequestHandler): def do_GET (self): self.send_response (100) self.end_headers () server = HTTPServer (('', 8000), NullHandler) server.handle_request() ------------------------------ Roy-Smiths-Computer:bug$ cat client.py #!/usr/bin/env python import urllib urllib.urlopen ('http://127.0.0.1:8000') --------------------------------- Roy-Smiths-Computer:bug$ ./client.py Traceback (most recent call last): File "./client.py", line 5, in ? urllib.urlopen ('http://127.0.0.1:8000') File "/usr/local/lib/python2.3/urllib.py", line 76, in urlopen return opener.open(url) File "/usr/local/lib/python2.3/urllib.py", line 181, in open return getattr(self, name)(url) File "/usr/local/lib/python2.3/urllib.py", line 306, in open_http return self.http_error(url, fp, errcode, errmsg, headers) File "/usr/local/lib/python2.3/urllib.py", line 323, in http_error return self.http_error_default(url, fp, errcode, errmsg, headers) File "/usr/local/lib/python2.3/urllib.py", line 551, in http_error_default return addinfourl(fp, headers, "http:" + url) File "/usr/local/lib/python2.3/urllib.py", line 837, in __init__ addbase.__init__(self, fp) File "/usr/local/lib/python2.3/urllib.py", line 787, in __init__ self.read = self.fp.read AttributeError: 'NoneType' object has no attribute 'read' --------------------------------- I'll give urllib2 a try and see how that works. ---------------------------------------------------------------------- Comment By: K Lars Lohn (lohnk) Date: 2005-03-16 12:07 Message: Logged In: YES user_id=1239273 I've changed over to urllib2. The only complication involved the exception handling model: urllib2's HTTPError exceptions cannot be pickled because they contain an open socket._fileobject. While mildly inconvenient, the workaround was not difficult. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2005-03-15 14:09 Message: Logged In: YES user_id=44345 Looking through the code I believe I traced the problem back to httplib.HTTP which sets self.fp to None when it's closed. It seems that urllib is trying to access this object after the connection's been closed. I realize the problem has passed for the moment, but have you considered using urllib2? The urllib library still uses httplib.HTTP which is really only there for backward compatibility. From this end it would be nice to leave urllib and httplib.HTTP alone. New apps should probably use urllib2 which uses the newer httplib.HTTPConnection class. ---------------------------------------------------------------------- Comment By: K Lars Lohn (lohnk) Date: 2005-03-15 11:50 Message: Logged In: YES user_id=1239273 This problem is apparently transient depending on network conditions or, perhaps, the configuration of the server end. On 3/14 the problem has mysteriously vanished.... ---------------------------------------------------------------------- Comment By: Jarek Zgoda (zgoda) Date: 2005-03-15 04:52 Message: Logged In: YES user_id=92222 No such error on Windows: Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163401&group_id=5470 From noreply at sourceforge.net Sun Apr 3 05:42:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 05:42:07 2005 Subject: [ python-Bugs-1112549 ] cgi.FieldStorage memory usage can spike in line-oriented ops Message-ID: Bugs item #1112549, was opened at 2005-01-30 08:40 Message generated for change (Comment added) made by chrism You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris McDonough (chrism) Assigned to: Nobody/Anonymous (nobody) Summary: cgi.FieldStorage memory usage can spike in line-oriented ops Initial Comment: Various parts of cgi.FieldStorage call its "read_lines_to_outerboundary", "read_lines" and "skip_lines" methods. These methods use the "readline" method of the file object that represents an input stream. The input stream is typically data supplied by an untrusted source (such as a user uploading a file from a web browser). The input data is not required by the RFC 822/1521/1522/1867 specifications to contain any newline characters. For example, it is within the bounds of the specification to supply a a multipart/form-data input stream with a "file-data" part that consists of a 2GB string composed entirely of "x" characters (which happens to be something I did that led me to noticing this bug). The simplest fix is to make use of the "size" argument of the readline method of the file object where it is used within all parts of FieldStorage that make use of it. A patch against the Python 2.3.4 cgi.py module that does this is attached. ---------------------------------------------------------------------- >Comment By: Chris McDonough (chrism) Date: 2005-04-02 22:42 Message: Logged In: YES user_id=32974 An updated test_cgi.py is attached. I test both the readline behavior and add a test for basic multipart parsing. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 23:48 Message: Logged In: YES user_id=6380 Can I tweak you into uploading a unit test? ---------------------------------------------------------------------- Comment By: Chris McDonough (chrism) Date: 2005-03-31 21:56 Message: Logged In: YES user_id=32974 Re: parse_multipart.. yes, it looks like there's no use fixing that as it just turns around and puts the line into a list.. it is vulnerable but just by virtue of its non-use of a tempfile, it appears doomed anyway for large requests. I don't know of anything that uses it. Good catch wrt boundary recognition bug, I'm uploading another patch. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 17:13 Message: Logged In: YES user_id=6380 Methinks that the fix isn't quite right: it would incorrectly recognize as a boundary a very long line starting with "--" followed by the appropriate random string at offset 2**16. This could probably be taken care of by adding a flag that is true initially and after that keeps track of whether the previous line ended in \n. Also, there's a call to fp.readline() in parse_multipart() that you didn't patch -- it wouldn't help because that code is saving the lines in a list anyway, but isn't that code vulnerable as well? Or is it not used? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 From noreply at sourceforge.net Sun Apr 3 06:00:06 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 06:00:08 2005 Subject: [ python-Bugs-1112549 ] cgi.FieldStorage memory usage can spike in line-oriented ops Message-ID: Bugs item #1112549, was opened at 2005-01-30 08:40 Message generated for change (Comment added) made by chrism You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris McDonough (chrism) Assigned to: Nobody/Anonymous (nobody) Summary: cgi.FieldStorage memory usage can spike in line-oriented ops Initial Comment: Various parts of cgi.FieldStorage call its "read_lines_to_outerboundary", "read_lines" and "skip_lines" methods. These methods use the "readline" method of the file object that represents an input stream. The input stream is typically data supplied by an untrusted source (such as a user uploading a file from a web browser). The input data is not required by the RFC 822/1521/1522/1867 specifications to contain any newline characters. For example, it is within the bounds of the specification to supply a a multipart/form-data input stream with a "file-data" part that consists of a 2GB string composed entirely of "x" characters (which happens to be something I did that led me to noticing this bug). The simplest fix is to make use of the "size" argument of the readline method of the file object where it is used within all parts of FieldStorage that make use of it. A patch against the Python 2.3.4 cgi.py module that does this is attached. ---------------------------------------------------------------------- >Comment By: Chris McDonough (chrism) Date: 2005-04-02 23:00 Message: Logged In: YES user_id=32974 FYI, I'd be happy to do the merging here if you wanted to give me checkin access. ---------------------------------------------------------------------- Comment By: Chris McDonough (chrism) Date: 2005-04-02 22:42 Message: Logged In: YES user_id=32974 An updated test_cgi.py is attached. I test both the readline behavior and add a test for basic multipart parsing. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 23:48 Message: Logged In: YES user_id=6380 Can I tweak you into uploading a unit test? ---------------------------------------------------------------------- Comment By: Chris McDonough (chrism) Date: 2005-03-31 21:56 Message: Logged In: YES user_id=32974 Re: parse_multipart.. yes, it looks like there's no use fixing that as it just turns around and puts the line into a list.. it is vulnerable but just by virtue of its non-use of a tempfile, it appears doomed anyway for large requests. I don't know of anything that uses it. Good catch wrt boundary recognition bug, I'm uploading another patch. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-03-31 17:13 Message: Logged In: YES user_id=6380 Methinks that the fix isn't quite right: it would incorrectly recognize as a boundary a very long line starting with "--" followed by the appropriate random string at offset 2**16. This could probably be taken care of by adding a flag that is true initially and after that keeps track of whether the previous line ended in \n. Also, there's a call to fp.readline() in parse_multipart() that you didn't patch -- it wouldn't help because that code is saving the lines in a list anyway, but isn't that code vulnerable as well? Or is it not used? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112549&group_id=5470 From noreply at sourceforge.net Sun Apr 3 10:48:10 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 10:48:14 2005 Subject: [ python-Feature Requests-1175686 ] add "reload" function Message-ID: Feature Requests item #1175686, was opened at 2005-04-03 08:48 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1175686&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: add "reload" function Initial Comment: The file menu in Editor windows could benefit from a "reload" function that re-reads the file from disc and loads it into the editor, replacing what's currently in the editor. That's useful for those of us who like to edit with Emacs in another window while running the code under Idle. To get really fancy, it might be cool to optionally display a diff between the in-editor version and the on-disk version allowing merging changes, but that's maybe over the top. Other editors usually do have commands like this, e.g.M-x Revert-file in Emacs or ":e!" in vi. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1175686&group_id=5470 From noreply at sourceforge.net Sun Apr 3 15:39:30 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 15:39:32 2005 Subject: [ python-Bugs-1174712 ] subclassing ModuleType and another built-in type Message-ID: Bugs item #1174712, was opened at 2005-04-01 10:22 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174712&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Armin Rigo (arigo) Assigned to: Nobody/Anonymous (nobody) Summary: subclassing ModuleType and another built-in type Initial Comment: class X(types.ModuleType, str): pass X('name') -> segfault This buggy subclassing goes through typeobject.c's checks because PyModuleObject looks exactly like a user-defined subclass of 'object': it has a PyObject_HEAD followed only by the dict, as specified by tp_dictoffset. A fix would be to forbid any subclassing to move the tp_dictoffset of a non-heap type. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-03 14:39 Message: Logged In: YES user_id=6656 > This might point to the need for cleaning up typeobject.c's > darker corners... No kidding. Maybe Samuele, you and I can gang up on Guido at EuroPython (is he sprinting? Perhaps we should ask). ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 13:27 Message: Logged In: YES user_id=4771 The proposed fix is not good enough. If another built-in C type similar to PyModuleObject is defined (say by a C extension module), then it would become possible to subclass both this new type and ModuleType. This might lead to unwanted behavior if e.g. one parent class allows rebinding __dict__ and the other not (and crashes if __dict__ is rebound). This might point to the need for cleaning up typeobject.c's darker corners... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174712&group_id=5470 From noreply at sourceforge.net Sun Apr 3 17:26:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 17:26:54 2005 Subject: [ python-Bugs-1175848 ] poorly named variable in urllib2.py Message-ID: Bugs item #1175848, was opened at 2005-04-03 11:26 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175848&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Roy Smith (roysmith) Assigned to: Nobody/Anonymous (nobody) Summary: poorly named variable in urllib2.py Initial Comment: This is kind of trivial, but in urllib2.OpenerDirector.__init__, the local variable "server_version" is poorly named. The name makes it sound like it's the version of the HTTP (or whatever) server we're talking to. How about client_version or library_version or module_version? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175848&group_id=5470 From noreply at sourceforge.net Sun Apr 3 18:24:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 18:24:38 2005 Subject: [ python-Bugs-1175202 ] python hangs if import statement launches threads Message-ID: Bugs item #1175202, was opened at 2005-04-02 07:24 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175202&group_id=5470 Category: Threads Group: Python 2.4 >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Jeff Stearns (jeffstearns) Assigned to: Nobody/Anonymous (nobody) Summary: python hangs if import statement launches threads Initial Comment: (This bug looks similar to bug 1175194, but reports a different problem.) I have a simple module which launches multiple HTTP client threads. The main thread creates 10 HTTP clients, each of which fetches documents from a web server. The main thread then simply goes to sleep while the client threads work. The threads are launched when the module is imported. If I launch the script via python bug1.py it launches and runs OK. But if I launch the script via python -c 'import bug1' it runs a bit, then hangs. Here's an example: jps@cure> ./python -c 'import bug1' Using 10 threads cccccccccc <- [program hangs here] Each thread prints a character every time that it does something interesting. The 'c' characters indicate that a connect syscall was made. These should be followed by 'C', indicating the the connect returned. That never happens. You might argue that it's inappropriate to launch threads from within import statement, but I can't find a specific prohibition against it. Here's a trace of the last few syscalls before it hangs (pids are actually thread ids): [pid 15481] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15481] futex(0x8148698, FUTEX_WAIT, 0, NULL [pid 15482] futex(0x81a9a80, FUTEX_WAKE, 1) = 0 [pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15482] write(2, "c", 1c) = 1 [pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15482] futex(0x820cc48, FUTEX_WAKE, 1) = 0 [pid 15482] futex(0x8148698, FUTEX_WAIT, 0, NULL <- hangs here Please note that this bug is similar to bug 1175194, but reports a different problem. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-03 18:24 Message: Logged In: YES user_id=21627 This is not a bug. The import statement returns when the code of the module completes. In this example, the import statement returns when the call run(10) completes. As the run method has a very long sleep, the import won't return until that sleep is over. So the problem is not that 10 threads are started, but that the module then blocks in its module execution. Furthermore, Python serializes all import statements, with a single import lock. So while one thread performs an import, other threads block when trying to import. So if the threads just started try to import something, they wait until the "main" import completes. If the main imports wait for the threads to complete (i.e. the commented-out thread.join), you get a dead-lock. Closing as "won't fix". ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175202&group_id=5470 From noreply at sourceforge.net Sun Apr 3 18:28:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 18:28:26 2005 Subject: [ python-Feature Requests-1155485 ] file() on a file Message-ID: Feature Requests item #1155485, was opened at 2005-03-03 00:48 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1155485&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Felix Lee (felixlee) Assigned to: Nobody/Anonymous (nobody) Summary: file() on a file Initial Comment: it would be nice if file(f) worked when f is already a file, either by returning f or by constructing a new file that refers to the same thing. that would make it easy to write functions that can take either a file or a filename as an argument, like so: def p(f): print list(file(f)) which is kind of like using int() as a cast operation. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-03 18:28 Message: Logged In: YES user_id=21627 Since several people have commented that the change would not be desirable, I'm closing it as "Won't fix". If you still think that feature should be added, please write a PEP, and collect feedback from the community (altough a part of the community will likely react the same way to such a PEP). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-03-15 01:00 Message: Logged In: YES user_id=31435 I'm also -1 on this -- I have no idea what would make sense when applying file() to a file object, or to a file-like object. Therefore anything it did in response would be surprising to me, except for raising an exception. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-03-14 22:20 Message: Logged In: YES user_id=21627 The same argument can *not* be made for strings, since strings are immutable, so it does not matter whether you get the original thing or a copy. Also, str(x) invokes x's __str__ conversion. For lists, the argument is also different: list(x) requires an iterable object and creates a list out of it. The notion of an iterable object is well-defined, and if you pass something that is not iterable, you get a TypeError. For files, this is entirely different. file() does not take one argument, but three (and two of them are optional). The first (mandatory) argument is a "filename". It might be debatable what precisely a file name is, and indeed you can pass both byte strings and unicode strings. A file, most certainly, is *not* a filename, and there is no notion of "converting to a file" in Python (e.g. by means of __file__). This just isn't a useful generalization. ---------------------------------------------------------------------- Comment By: Felix Lee (felixlee) Date: 2005-03-14 21:47 Message: Logged In: YES user_id=77867 that argument also works against str() and list(). it's not obvious whether str(s) and list(v) should return itself, a proxy, or a copy, and they're all useful in different situations. I don't think anyone would argue that the ambiguity means those should be undefined. I think file(f) is similar. it has a few more issues because files are special, but I think picking a reasonable behavior for file(f) would simplify some programming. returning a dup is probably the least surprising in most situations, because of f.close(). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-03-13 23:53 Message: Logged In: YES user_id=21627 It's not at all obvious what this should do. Three alternatives come to mind: 1. return f 2. return a wrapper object which delegates all calls 3. return a new file object, which is dup(2)'ed from the original one. Either of them might be meaningful in some context, and they are mutually incompatible. In the face of ambiguity, refuse the temptation to guess, so I'm -1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1155485&group_id=5470 From noreply at sourceforge.net Sun Apr 3 22:20:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 22:20:58 2005 Subject: [ python-Bugs-1175967 ] StringIO and cStringIO don't provide 'name' attribute Message-ID: Bugs item #1175967, was opened at 2005-04-03 15:20 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175967&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: logistix (logistix) Assigned to: Nobody/Anonymous (nobody) Summary: StringIO and cStringIO don't provide 'name' attribute Initial Comment: Documentation explicitly states that file-like objects should return a repr-style psuedoname. Patch is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175967&group_id=5470 From noreply at sourceforge.net Sun Apr 3 23:21:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 3 23:21:22 2005 Subject: [ python-Bugs-1176012 ] compiler module didn't get updated for "class foo():pass" Message-ID: Bugs item #1176012, was opened at 2005-04-03 16:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: logistix (logistix) Assigned to: Nobody/Anonymous (nobody) Summary: compiler module didn't get updated for "class foo():pass" Initial Comment: Patch with fix and testcase is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 From noreply at sourceforge.net Mon Apr 4 00:01:51 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 4 00:01:54 2005 Subject: [ python-Bugs-1176012 ] compiler module didn't get updated for "class foo():pass" Message-ID: Bugs item #1176012, was opened at 2005-04-03 16:21 Message generated for change (Comment added) made by logistix You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: logistix (logistix) Assigned to: Nobody/Anonymous (nobody) Summary: compiler module didn't get updated for "class foo():pass" Initial Comment: Patch with fix and testcase is attached. ---------------------------------------------------------------------- >Comment By: logistix (logistix) Date: 2005-04-03 17:01 Message: Logged In: YES user_id=699438 The parser module didn't get corrected either. Here is another patch with test case for that. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 From noreply at sourceforge.net Mon Apr 4 18:29:07 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 4 18:29:09 2005 Subject: [ python-Bugs-1176467 ] Python garbage collector isn't detecting deadlocks Message-ID: Bugs item #1176467, was opened at 2005-04-04 09:29 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176467&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nathan Marushak (nmarushak) Assigned to: Nobody/Anonymous (nobody) Summary: Python garbage collector isn't detecting deadlocks Initial Comment: BACKGROUND INFORMATION: Python version: Python 2.3.3 (#1, May 7 2004, 10:31:40) Operating System: Linux 2.6.7-1.494.2.2smp #1 SMP Tue Aug 3 09:59:49 EDT 2004 i686 i686 i386 GNU/Linux PROBLEM: The 2.3.3 garbage collector does not appear to detect deadlocks that occur when 2 objects reference one another. I checked the existing bug list, albeit briefly, to see if a similar bug had already been entered. I did not find an existing entry, but I apologize if this is a duplicate. I've attached a short example output file that demonstrates the issue. It would be beneficial if the "gc" module could detect when 2 objects have no other outstanding references except to one another. If modifying gc is not possible for some reason, then updating the Python gc documentation to illustrate this issue would be helpful. WORKAROUND: The problem can be worked around by ensuring each object removes the circular reference when it no longer needs to refer to the other object. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176467&group_id=5470 From noreply at sourceforge.net Mon Apr 4 20:22:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 4 20:22:27 2005 Subject: [ python-Bugs-1176467 ] Python garbage collector isn't detecting deadlocks Message-ID: Bugs item #1176467, was opened at 2005-04-04 16:29 Message generated for change (Comment added) made by nascheme You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176467&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nathan Marushak (nmarushak) >Assigned to: Neil Schemenauer (nascheme) Summary: Python garbage collector isn't detecting deadlocks Initial Comment: BACKGROUND INFORMATION: Python version: Python 2.3.3 (#1, May 7 2004, 10:31:40) Operating System: Linux 2.6.7-1.494.2.2smp #1 SMP Tue Aug 3 09:59:49 EDT 2004 i686 i686 i386 GNU/Linux PROBLEM: The 2.3.3 garbage collector does not appear to detect deadlocks that occur when 2 objects reference one another. I checked the existing bug list, albeit briefly, to see if a similar bug had already been entered. I did not find an existing entry, but I apologize if this is a duplicate. I've attached a short example output file that demonstrates the issue. It would be beneficial if the "gc" module could detect when 2 objects have no other outstanding references except to one another. If modifying gc is not possible for some reason, then updating the Python gc documentation to illustrate this issue would be helpful. WORKAROUND: The problem can be worked around by ensuring each object removes the circular reference when it no longer needs to refer to the other object. ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2005-04-04 18:22 Message: Logged In: YES user_id=35752 The gc is working as intended and as documented. Please refer to the gc module documentation, especially the section on the "garbage" attribute. Your objects are not collected because they have __del__ methods. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176467&group_id=5470 From noreply at sourceforge.net Tue Apr 5 00:01:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 00:01:19 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 17:14 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-05 00:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Tue Apr 5 04:12:55 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 04:12:58 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 02:05 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Carroll (vmlinuxz) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-04 19:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Tue Apr 5 05:37:04 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 05:37:07 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 05:05 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Carroll (vmlinuxz) >Assigned to: Raymond Hettinger (rhettinger) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-04 21:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Tue Apr 5 05:37:15 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 05:37:19 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 02:05 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Carroll (vmlinuxz) Assigned to: Raymond Hettinger (rhettinger) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-04 20:37 Message: Logged In: YES user_id=971153 It turns out that the code in plain text bug report is more or less readable, so I can see what you were trying to do... Seems like you are doing ftp.dir("LIST "+ftpDir) But ftplib docs: http://www.python.org/doc/current/lib/ftp-objects.html say that you need to specify the directory ftp.dir(ftpDir) so it seems that you are using ftp.dir() incorrectly.... But, even then IndexError seems like a wrong kind of exception for this kind of error... So, I guess, my original request for a working code snippet which would allow to reproduce the bug still stands.. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-04 19:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Tue Apr 5 06:31:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 06:31:44 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 05:05 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Carroll (vmlinuxz) >Assigned to: Nobody/Anonymous (nobody) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-04 23:31 Message: Logged In: YES user_id=80475 I fixed the IndexError. It should have been a protocol error. See Lib/ftplib.py 1.74. Leaving the bug open so the OP can make a more detailed post so we can see what the root issue is about. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-04 22:37 Message: Logged In: YES user_id=971153 It turns out that the code in plain text bug report is more or less readable, so I can see what you were trying to do... Seems like you are doing ftp.dir("LIST "+ftpDir) But ftplib docs: http://www.python.org/doc/current/lib/ftp-objects.html say that you need to specify the directory ftp.dir(ftpDir) so it seems that you are using ftp.dir() incorrectly.... But, even then IndexError seems like a wrong kind of exception for this kind of error... So, I guess, my original request for a working code snippet which would allow to reproduce the bug still stands.. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-04 21:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Tue Apr 5 10:50:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 10:50:50 2005 Subject: [ python-Bugs-1176893 ] Readline segfault Message-ID: Bugs item #1176893, was opened at 2005-04-05 10:50 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Walter D?rwald (doerwalter) Assigned to: Michael Hudson (mwh) Summary: Readline segfault Initial Comment: The latest change to the readline module has broken tab completion: ./python Python 2.5a0 (#1, Apr 5 2005, 01:14:33) [GCC 3.3.5 (Gentoo Linux 3.3.5-r1, ssp-3.3.2-3, pie-8.7.7.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline, rlcompleter [25913 refs] >>> readline.parse_and_bind("tab: complete") [25913 refs] >>> Segmentation fault [Press tab after the parse_and_bind() call] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 From noreply at sourceforge.net Tue Apr 5 12:18:05 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 12:18:08 2005 Subject: [ python-Bugs-1176893 ] Readline segfault Message-ID: Bugs item #1176893, was opened at 2005-04-05 09:50 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Walter D?rwald (doerwalter) Assigned to: Michael Hudson (mwh) Summary: Readline segfault Initial Comment: The latest change to the readline module has broken tab completion: ./python Python 2.5a0 (#1, Apr 5 2005, 01:14:33) [GCC 3.3.5 (Gentoo Linux 3.3.5-r1, ssp-3.3.2-3, pie-8.7.7.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline, rlcompleter [25913 refs] >>> readline.parse_and_bind("tab: complete") [25913 refs] >>> Segmentation fault [Press tab after the parse_and_bind() call] ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-05 11:18 Message: Logged In: YES user_id=6656 I'm going to go out on a limb and suggest this is a bug in the PyGilState_ functions. The problem burrows down to calling PyThread_release_lock(interpreter_lock) when interpreter_lock is NULL, i.e. PyEval_InitThreads hasn't been called. (if you start a thread before running the crashing code, it doesn't crash, because the GIL has been allocated). Not sure what the solution is, or who to bug (time to read cvs log, I guess). A silly workaround is to put PyEval_InitThreads in initreadline. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 From noreply at sourceforge.net Tue Apr 5 12:27:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 12:27:06 2005 Subject: [ python-Bugs-1176893 ] Readline segfault Message-ID: Bugs item #1176893, was opened at 2005-04-05 09:50 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Walter D?rwald (doerwalter) Assigned to: Michael Hudson (mwh) Summary: Readline segfault Initial Comment: The latest change to the readline module has broken tab completion: ./python Python 2.5a0 (#1, Apr 5 2005, 01:14:33) [GCC 3.3.5 (Gentoo Linux 3.3.5-r1, ssp-3.3.2-3, pie-8.7.7.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline, rlcompleter [25913 refs] >>> readline.parse_and_bind("tab: complete") [25913 refs] >>> Segmentation fault [Press tab after the parse_and_bind() call] ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-05 11:27 Message: Logged In: YES user_id=6656 Or maybe this one line patch is the answer (it certainly fixes this case). Tim, can you spare a minute to think about this? The patch simply adds a check to PyEval_ReleaseThread that doesn't call PyThread_release_lock if the GIL hasn't been allocated. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-05 11:18 Message: Logged In: YES user_id=6656 I'm going to go out on a limb and suggest this is a bug in the PyGilState_ functions. The problem burrows down to calling PyThread_release_lock(interpreter_lock) when interpreter_lock is NULL, i.e. PyEval_InitThreads hasn't been called. (if you start a thread before running the crashing code, it doesn't crash, because the GIL has been allocated). Not sure what the solution is, or who to bug (time to read cvs log, I guess). A silly workaround is to put PyEval_InitThreads in initreadline. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 From noreply at sourceforge.net Tue Apr 5 12:27:21 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 12:27:25 2005 Subject: [ python-Bugs-1176893 ] Readline segfault Message-ID: Bugs item #1176893, was opened at 2005-04-05 09:50 Message generated for change (Settings changed) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Walter D?rwald (doerwalter) >Assigned to: Tim Peters (tim_one) Summary: Readline segfault Initial Comment: The latest change to the readline module has broken tab completion: ./python Python 2.5a0 (#1, Apr 5 2005, 01:14:33) [GCC 3.3.5 (Gentoo Linux 3.3.5-r1, ssp-3.3.2-3, pie-8.7.7.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline, rlcompleter [25913 refs] >>> readline.parse_and_bind("tab: complete") [25913 refs] >>> Segmentation fault [Press tab after the parse_and_bind() call] ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-05 11:27 Message: Logged In: YES user_id=6656 Or maybe this one line patch is the answer (it certainly fixes this case). Tim, can you spare a minute to think about this? The patch simply adds a check to PyEval_ReleaseThread that doesn't call PyThread_release_lock if the GIL hasn't been allocated. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-05 11:18 Message: Logged In: YES user_id=6656 I'm going to go out on a limb and suggest this is a bug in the PyGilState_ functions. The problem burrows down to calling PyThread_release_lock(interpreter_lock) when interpreter_lock is NULL, i.e. PyEval_InitThreads hasn't been called. (if you start a thread before running the crashing code, it doesn't crash, because the GIL has been allocated). Not sure what the solution is, or who to bug (time to read cvs log, I guess). A silly workaround is to put PyEval_InitThreads in initreadline. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 From noreply at sourceforge.net Tue Apr 5 15:56:10 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 5 15:56:13 2005 Subject: [ python-Bugs-1177077 ] [PyPI] Password reset problem. Message-ID: Bugs item #1177077, was opened at 2005-04-05 13:56 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177077&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Darek Suchojad (dsuch) Assigned to: Nobody/Anonymous (nobody) Summary: [PyPI] Password reset problem. Initial Comment: Hello, the URL for reseting a password to PyPI is http://www.python.org/pypi?:action=password_reset&email=my@email. However, this page yields a message """ Error... There's been a problem with your request psycopg.ProgrammingError: ERROR: syntax error at or near "where" at character 104 update users set password='6c2105e62b35507733ee49fdaed9815022b324e6', email='my@email', where name='myname' """ Clearly, there's a superfluous comma before 'where name='. I'm filling this bug report per request from webmaster'python'org. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177077&group_id=5470 From noreply at sourceforge.net Wed Apr 6 03:03:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 03:03:34 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 18:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 6 03:06:05 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 03:06:08 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 18:03 Message generated for change (Comment added) made by majid You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Fazal Majid (majid) Date: 2005-04-05 18:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 6 04:52:42 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 04:52:44 2005 Subject: [ python-Bugs-1161595 ] Minor error in section 3.2 Message-ID: Bugs item #1161595, was opened at 2005-03-11 11:29 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1161595&group_id=5470 Category: Documentation Group: None Status: Open Resolution: Invalid Priority: 1 Submitted By: Jeremy Barbay (jyby) Assigned to: Nobody/Anonymous (nobody) Summary: Minor error in section 3.2 Initial Comment: In the section "3.2 First Steps Towards Programming " of the Python tutorial (http://docs.python.org/tut/node5.html), the output of both implementations of the Fibonacci sequence computation is incorrect. As written, only one 1 should be output. You should either remove one 1 from the input, or replace the lines "print b" and "print b," by "print a" and "print a,". This is minor but might confuse unnecessarily beginners. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 19:52 Message: Logged In: YES user_id=971153 It indeed seems that the output in tutorial is correct could you close or delete the bug then? Thanks ---------------------------------------------------------------------- Comment By: Jeremy Barbay (jyby) Date: 2005-03-11 11:40 Message: Logged In: YES user_id=149696 All my apologies: I didn't check my code correctly: as the algorithm is initializing a with 0 instead of 1, the output is correct. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1161595&group_id=5470 From noreply at sourceforge.net Wed Apr 6 05:11:42 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 05:11:44 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-06 01:03 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 01:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 6 05:24:18 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 05:24:21 2005 Subject: [ python-Bugs-1177077 ] [PyPI] Password reset problem. Message-ID: Bugs item #1177077, was opened at 2005-04-05 13:56 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177077&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Darek Suchojad (dsuch) Assigned to: Nobody/Anonymous (nobody) Summary: [PyPI] Password reset problem. Initial Comment: Hello, the URL for reseting a password to PyPI is http://www.python.org/pypi?:action=password_reset&email=my@email. However, this page yields a message """ Error... There's been a problem with your request psycopg.ProgrammingError: ERROR: syntax error at or near "where" at character 104 update users set password='6c2105e62b35507733ee49fdaed9815022b324e6', email='my@email', where name='myname' """ Clearly, there's a superfluous comma before 'where name='. I'm filling this bug report per request from webmaster'python'org. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:24 Message: Logged In: YES user_id=81797 I know Brett told you to submit it here, but I can't imagine why that would be more appropriate than using your first instinct and submitting it to the pypi tracker. It looks like the bug is on line 602 of "store.py", change ", where" to " where" Sean ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177077&group_id=5470 From noreply at sourceforge.net Wed Apr 6 05:33:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 05:33:53 2005 Subject: [ python-Bugs-1175022 ] property example code error Message-ID: Bugs item #1175022, was opened at 2005-04-01 20:09 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175022&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: John Ridley (ojokimu) Assigned to: Nobody/Anonymous (nobody) Summary: property example code error Initial Comment: The example code for 'property' in lib/built-in-funcs.html may produce an error if run "as is": Python 2.4.1 (#1, Mar 31 2005, 21:33:58) [GCC 3.4.1 (Mandrakelinux (Alpha 3.4.1-3mdk)] on linux2 >>> class C(object): ... def getx(self): return self.__x ... def setx(self, value): self.__x = value ... def delx(self): del self.__x ... x = property(getx, setx, delx, "I'm the 'x' property.") ... >>> c=C() >>> c.x Traceback (most recent call last): File "", line 1, in ? File "", line 2, in getx AttributeError: 'C' object has no attribute '_C__x' The same goes for 'del c.x' (although not 'c.x = 0', of course). A more "typical" way of defining managed attributes would be to include an 'init' as follows: class C(object): def __init__(self): self.__x = None def getx(self): return self.__x def setx(self, value): self.__x = value def delx(self): del self.__x x = property(getx, setx, delx, "I'm the 'x' property.") ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:33 Message: Logged In: YES user_id=81797 I agree, adding the __init__ to set a value would be useful. +1 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175022&group_id=5470 From noreply at sourceforge.net Wed Apr 6 05:39:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 05:39:06 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 04:48 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:39 Message: Logged In: YES user_id=81797 I am able to reproduce this on a Fedora Core 3 Linux system: >>> fp = open('/dev/zero', 'rb') >>> d = fp.read() Traceback (most recent call last): File "", line 1, in ? MemoryError >>> print os.stat('/dev/zero').st_size 0 What about only trusting st_size if the file is a regular file, not a directory or other type of special file? Sean ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 12:31 Message: Logged In: YES user_id=4771 os.stat() doesn't always give consistent results on dev files. On my machine for some reason os.stat('/dev/null') appears to be random (and extremely large). I suspect that on the OP's machine os.stat('/dev/zero') is not 0 either, but a random number that turns out to be negative, hence a "bad argument" SystemError. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 21:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 09:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Wed Apr 6 08:40:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 08:40:56 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 06:48 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 08:40 Message: Logged In: YES user_id=21627 The problem is different. Instead, _PyString_Resize complains that the new buffersize of the string is negative. This in turn happens because the string manages to get larger >2GB, which in turn happens because buffersize is size_t, yet _PyString_Resize expects int. I don't know how Linux manages to allocate such a large string without thrashing. There is a minor confusion with stat() as well: new_buffersize tries to find out how much bytes are left to the end of the file. In the case of /dev/zero, both fstat and lseek are "lying" by returning 0. As lseek returns 0, ftell is invoked and returns non-zero. Then, newbuffer does not trust the values, and just adds BIGCHUNK. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:39 Message: Logged In: YES user_id=81797 I am able to reproduce this on a Fedora Core 3 Linux system: >>> fp = open('/dev/zero', 'rb') >>> d = fp.read() Traceback (most recent call last): File "", line 1, in ? MemoryError >>> print os.stat('/dev/zero').st_size 0 What about only trusting st_size if the file is a regular file, not a directory or other type of special file? Sean ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 14:31 Message: Logged In: YES user_id=4771 os.stat() doesn't always give consistent results on dev files. On my machine for some reason os.stat('/dev/null') appears to be random (and extremely large). I suspect that on the OP's machine os.stat('/dev/zero') is not 0 either, but a random number that turns out to be negative, hence a "bad argument" SystemError. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 23:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 11:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Wed Apr 6 08:52:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 08:52:26 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 04:48 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 06:52 Message: Logged In: YES user_id=81797 Linux can do a very fast allocation if it has swap available. It reserves space, but does not actually assign the memory until you try to use it. In my case, I have 1GB of RAM, around 700MB free, and another 2GB in swap. So, I have plenty unless I use it. In C I can malloc 1GB and unless I write every page in that block the system doesn't really give the pages to the process. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 06:40 Message: Logged In: YES user_id=21627 The problem is different. Instead, _PyString_Resize complains that the new buffersize of the string is negative. This in turn happens because the string manages to get larger >2GB, which in turn happens because buffersize is size_t, yet _PyString_Resize expects int. I don't know how Linux manages to allocate such a large string without thrashing. There is a minor confusion with stat() as well: new_buffersize tries to find out how much bytes are left to the end of the file. In the case of /dev/zero, both fstat and lseek are "lying" by returning 0. As lseek returns 0, ftell is invoked and returns non-zero. Then, newbuffer does not trust the values, and just adds BIGCHUNK. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:39 Message: Logged In: YES user_id=81797 I am able to reproduce this on a Fedora Core 3 Linux system: >>> fp = open('/dev/zero', 'rb') >>> d = fp.read() Traceback (most recent call last): File "", line 1, in ? MemoryError >>> print os.stat('/dev/zero').st_size 0 What about only trusting st_size if the file is a regular file, not a directory or other type of special file? Sean ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 12:31 Message: Logged In: YES user_id=4771 os.stat() doesn't always give consistent results on dev files. On my machine for some reason os.stat('/dev/null') appears to be random (and extremely large). I suspect that on the OP's machine os.stat('/dev/zero') is not 0 either, but a random number that turns out to be negative, hence a "bad argument" SystemError. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 21:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 09:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Wed Apr 6 09:06:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 09:06:57 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 06:48 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 09:06 Message: Logged In: YES user_id=21627 The surprising aspect is that the memory *is* being used. Python allocates 2GB of memory, and then passes this to read(2) (through stdio) to fill it with the contents of /dev/zero. This should cause a write operation to the memory pages, which in turn should cause them to consume actual memory. For some reason, this doesn't happen. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 08:52 Message: Logged In: YES user_id=81797 Linux can do a very fast allocation if it has swap available. It reserves space, but does not actually assign the memory until you try to use it. In my case, I have 1GB of RAM, around 700MB free, and another 2GB in swap. So, I have plenty unless I use it. In C I can malloc 1GB and unless I write every page in that block the system doesn't really give the pages to the process. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 08:40 Message: Logged In: YES user_id=21627 The problem is different. Instead, _PyString_Resize complains that the new buffersize of the string is negative. This in turn happens because the string manages to get larger >2GB, which in turn happens because buffersize is size_t, yet _PyString_Resize expects int. I don't know how Linux manages to allocate such a large string without thrashing. There is a minor confusion with stat() as well: new_buffersize tries to find out how much bytes are left to the end of the file. In the case of /dev/zero, both fstat and lseek are "lying" by returning 0. As lseek returns 0, ftell is invoked and returns non-zero. Then, newbuffer does not trust the values, and just adds BIGCHUNK. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:39 Message: Logged In: YES user_id=81797 I am able to reproduce this on a Fedora Core 3 Linux system: >>> fp = open('/dev/zero', 'rb') >>> d = fp.read() Traceback (most recent call last): File "", line 1, in ? MemoryError >>> print os.stat('/dev/zero').st_size 0 What about only trusting st_size if the file is a regular file, not a directory or other type of special file? Sean ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 14:31 Message: Logged In: YES user_id=4771 os.stat() doesn't always give consistent results on dev files. On my machine for some reason os.stat('/dev/null') appears to be random (and extremely large). I suspect that on the OP's machine os.stat('/dev/zero') is not 0 either, but a random number that turns out to be negative, hence a "bad argument" SystemError. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 23:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 11:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Wed Apr 6 09:11:13 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 09:11:16 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-06 03:03 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 09:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 03:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 6 09:17:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 09:17:21 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-04-01 04:48 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 07:17 Message: Logged In: YES user_id=81797 I'm quite sure that the 2GB is not getting filled when doing this. After running the first command, and checking /proc/meminfo, I see that only 46MB is shown as free, which means that there was no more than this amount of RAM consumed. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 07:06 Message: Logged In: YES user_id=21627 The surprising aspect is that the memory *is* being used. Python allocates 2GB of memory, and then passes this to read(2) (through stdio) to fill it with the contents of /dev/zero. This should cause a write operation to the memory pages, which in turn should cause them to consume actual memory. For some reason, this doesn't happen. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 06:52 Message: Logged In: YES user_id=81797 Linux can do a very fast allocation if it has swap available. It reserves space, but does not actually assign the memory until you try to use it. In my case, I have 1GB of RAM, around 700MB free, and another 2GB in swap. So, I have plenty unless I use it. In C I can malloc 1GB and unless I write every page in that block the system doesn't really give the pages to the process. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 06:40 Message: Logged In: YES user_id=21627 The problem is different. Instead, _PyString_Resize complains that the new buffersize of the string is negative. This in turn happens because the string manages to get larger >2GB, which in turn happens because buffersize is size_t, yet _PyString_Resize expects int. I don't know how Linux manages to allocate such a large string without thrashing. There is a minor confusion with stat() as well: new_buffersize tries to find out how much bytes are left to the end of the file. In the case of /dev/zero, both fstat and lseek are "lying" by returning 0. As lseek returns 0, ftell is invoked and returns non-zero. Then, newbuffer does not trust the values, and just adds BIGCHUNK. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:39 Message: Logged In: YES user_id=81797 I am able to reproduce this on a Fedora Core 3 Linux system: >>> fp = open('/dev/zero', 'rb') >>> d = fp.read() Traceback (most recent call last): File "", line 1, in ? MemoryError >>> print os.stat('/dev/zero').st_size 0 What about only trusting st_size if the file is a regular file, not a directory or other type of special file? Sean ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 12:31 Message: Logged In: YES user_id=4771 os.stat() doesn't always give consistent results on dev files. On my machine for some reason os.stat('/dev/null') appears to be random (and extremely large). I suspect that on the OP's machine os.stat('/dev/zero') is not 0 either, but a random number that turns out to be negative, hence a "bad argument" SystemError. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 21:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 09:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Wed Apr 6 09:20:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 09:20:38 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-06 01:03 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 07:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 07:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 01:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 6 09:27:00 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 09:27:03 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 18:03 Message generated for change (Comment added) made by majid You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Fazal Majid (majid) Date: 2005-04-06 00:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 00:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 00:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 20:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-05 18:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 6 11:04:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 11:04:23 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-06 01:03 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 09:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 07:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 07:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 07:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 01:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 6 12:59:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 12:59:51 2005 Subject: [ python-Bugs-1177674 ] error locale.getlocale() with LANGUAGE=eu_ES Message-ID: Bugs item #1177674, was opened at 2005-04-06 10:59 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177674&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zunbeltz Izaola (zunbeltz) Assigned to: Nobody/Anonymous (nobody) Summary: error locale.getlocale() with LANGUAGE=eu_ES Initial Comment: My LANGUAGE is set tu eu_ES but the getlocale() output is (None, None). This is check with the command python -c "import locale; print locale.getlocale()" python version 2.4 os. linux - ubuntu ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177674&group_id=5470 From noreply at sourceforge.net Wed Apr 6 14:39:26 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 14:39:31 2005 Subject: [ python-Bugs-1168135 ] Python 2.5a0 Tutorial errors and observations Message-ID: Bugs item #1168135, was opened at 2005-03-21 22:58 Message generated for change (Comment added) made by mrbax You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168135&group_id=5470 Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Michael R Bax (mrbax) Assigned to: Raymond Hettinger (rhettinger) Summary: Python 2.5a0 Tutorial errors and observations Initial Comment: Please find attached my updated list of errors and observations in response to Python 2.5a0. ---------------------------------------------------------------------- >Comment By: Michael R Bax (mrbax) Date: 2005-04-06 05:39 Message: Logged In: YES user_id=1055057 Thanks for your comments. A handful of meta-observations: Front Matter > Your first correction is wrong. -ly adverbs are never hyphenated > (Chicago Manual of Style, Table 6.1, for instance). "Wrong" is wrong. In fact, the CMS Q&A explicitly states that "it has long been the practice elsewhere -- among British writers, for example -- to hyphenate ly + participle/adjective compounds. ... So it is a matter not of who is right or wrong but of whose rule you are going to follow." But I'm happy to leave it as is, given that it is a regional preference. 3.1.2 > > >>> word[:1] = 'Splat' > > -- This is trying to assign 5 letters to one? > > Slice assignment is replacement, not overwriting. This is > replacing 1 byte with 5, which *is* valid, and perhaps the point > being made. Perhaps you would recommend another change to > be clearer? I'm not recommending a change per se; I'm showing what a newbie reader thinks! :-) 5.2 > > There is a way to remove an item from a list given its index > > instead of its value: the del statement. > > -- How is this different to pop? > > pop, builtin on top of del, also saves and returns the deleted value > so it can be bound to something, which takes longer. ie > def pop(self, i): > ret = self[i] > del self[i] > return ret Again, this is a question that the newbie reader will pose. I may know the answer, but I am not asking the question for myself. I think the question should be answered pre- emptively in the tutorial! 9.2 > > Otherwise, all variables found outside of the innermost scope > > are read-only. > > -- Explain what happens when you try to assign to a > > read-only variable? > > You create a new local of the same name and will not be able to > read the masked variable. Right -- again, this is for the benefit of the newbie. Let's put that in the tutorial! :-) ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-03-30 13:01 Message: Logged In: YES user_id=593130 Subject to reading that particular version (.5a0), I generally agree with your suggestions. Here are some specific comments on your comments. Feel free to incorporate them into a revised suggestion list if you wish. ----------- Your first correction is wrong. -ly adverbs are never hyphenated (Chicago Manual of Style, Table 6.1, for instance). ----------- 3.1.2 >>> word[:1] = 'Splat' -- This is trying to assign 5 letters to one? Slice assignment is replacement, not overwriting. This is replacing 1 byte with 5, which *is* valid, and perhaps the point being made. Perhaps you would recommend another change to be clearer? ----------- ##5.1.3 ##Combining these two special cases, we see that "map(None, list1, list2)" is a convenient way of turning a pair of lists into a list of pairs # -- Shouldn't one rather use zip()? I would agree that 'convenient' should be removed and a note added that this has been superceded by zip unless one wants the different behavior of extending shorter sequences. ---------- 5.1.3 filter(function, sequence)" returns a sequence (of the same type, if possible) -- How could this ever be impossible? I suppose a broken class, but then what would filter do? If filter 'works' for all builtins, I agree that we should just say so. Perhaps 'returns a sequence of the same type (for all builtins and sensible user classes)' -- if that is true -------- 5.2 There is a way to remove an item from a list given its index instead of its value: the del statement. -- How is this different to pop? pop, builtin on top of del, also saves and returns the deleted value so it can be bound to something, which takes longer. ie def pop(self, i): ret = self[i] del self[i] return ret ---------------------------- 5.3 Sequence unpacking requires that the list of variables on the left have the same number of elements as the length of the sequence -- requires that the list of variables on the left has (grammar) -- requires the list of variables on the left to have (alternate) Since the code sequence on the left is not a Python list but only a common-meaning list, I think even better would be "Sequence unpacking requires that the number of variables on the left be the same as the length of the sequence on the right" ------------------ 5.7 Comparisons may be combined by the Boolean operators and and or -- combined using the (style) In general, the return value of a short-circuit operator, when used as a general value and not as a Boolean, is the last evaluated argument. -- rewrite: When used as a general value and not as a Boolean, the return value of a short-circuit operator is the last evaluated argument. I would personally be a bit more radical here. As has been said often on c.l.p in response to newby misconceptions about 'and' and 'or', they are not operators, in the sense of syntactically abbreviating function calls, but flow-control keywords abbreviating if-else constructs. Saying so in the tutorial might 'short-circuit' some of the misconceptions. ------------- 9 a derived class can override any methods of its base class or classes, a method can call the method of a base class with the same name. -- classes, and a method (last phrase in a list) There are no special constructors or destructors. -- What about __init__, or is that a "normal" constructor? The special method __new__ constructs (and initializes immutables) (so why isn't it a 'special constructor'?), __init__ initializes, But as I remember, the quoted sentence does seem confusing. --------------- 9.2 Otherwise, all variables found outside of the innermost scope are read-only. -- Explain what happens when you try to assign to a read-only variable? You create a new local of the same name and will not be able to read the masked variable. ------------ 9.8 raise Class, instance -- What's the point? In the example thereafter, these three have the same effect: raise c() raise B, c() raise instance -- wasn't this "new form" covered in 8.5 (raise MyClass (2*2))? I personally agree that the tutorial should emphasize the newest form (raise instance) and only mention back-compatible forms in passing. ... - What is an example of an exception that is NOT a class? Last I knew, strings are still legal, though deprecated. ------------- 9.11 sum(i*i for i in range(10)) -- Why does this work?! because generator expressions require their own parentheses *unless* already directly enclosed by function-call parens. I agree that this nice quirk should be explained, ------------- B.1 ... >>> 7205759403792794L * 10L**30 / 2L**56 -- The "L" extension does not appear to be necessary for the input -- why is it used? Used to be needed before int-long unification. I agree that tutorial should be updated to future-oriented present. ---------- C The socket module uses the functions, getaddrinfo, and getnameinfo -- remove comma after "functions" [grammar] and after getaddrinfo ------------- ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168135&group_id=5470 From noreply at sourceforge.net Wed Apr 6 15:08:37 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 15:08:41 2005 Subject: [ python-Bugs-1177674 ] error locale.getlocale() with LANGUAGE=eu_ES Message-ID: Bugs item #1177674, was opened at 2005-04-06 19:59 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177674&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Zunbeltz Izaola (zunbeltz) Assigned to: Nobody/Anonymous (nobody) Summary: error locale.getlocale() with LANGUAGE=eu_ES Initial Comment: My LANGUAGE is set tu eu_ES but the getlocale() output is (None, None). This is check with the command python -c "import locale; print locale.getlocale()" python version 2.4 os. linux - ubuntu ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2005-04-06 22:08 Message: Logged In: YES user_id=55188 It's an intended behavior and conforms POSIX standard. locale doesn't set until program calls locale.setlocale() explicitly. miffy(perky):~% LC_ALL=ko_KR.UTF-8 python Python 2.4 (#2, Feb 4 2005, 12:07:54) [GCC 3.4.2 [FreeBSD] 20040728] on freebsd5 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.getlocale() (None, None) >>> locale.setlocale(locale.LC_ALL, '') 'ko_KR.UTF-8' >>> locale.getlocale() ['ko_KR', 'utf'] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177674&group_id=5470 From noreply at sourceforge.net Wed Apr 6 15:42:21 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 15:42:24 2005 Subject: [ python-Bugs-1177674 ] error locale.getlocale() with LANGUAGE=eu_ES Message-ID: Bugs item #1177674, was opened at 2005-04-06 10:59 Message generated for change (Comment added) made by zunbeltz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177674&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Zunbeltz Izaola (zunbeltz) Assigned to: Nobody/Anonymous (nobody) Summary: error locale.getlocale() with LANGUAGE=eu_ES Initial Comment: My LANGUAGE is set tu eu_ES but the getlocale() output is (None, None). This is check with the command python -c "import locale; print locale.getlocale()" python version 2.4 os. linux - ubuntu ---------------------------------------------------------------------- >Comment By: Zunbeltz Izaola (zunbeltz) Date: 2005-04-06 13:42 Message: Logged In: YES user_id=1139116 Sorry. The example is wrong. If I execute the comands you show (setlocale() and them getlocale()) I get the following error Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.4/locale.py", line 365, in getlocale return _parse_localename(localename) File "/usr/lib/python2.4/locale.py", line 278, in _parse_localename raise ValueError, 'unknown locale: %s' % localename ValueError: unknown locale: eu_ES (I find this error when i want to use pybliographic and it use setlocale before getlocale) ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2005-04-06 13:08 Message: Logged In: YES user_id=55188 It's an intended behavior and conforms POSIX standard. locale doesn't set until program calls locale.setlocale() explicitly. miffy(perky):~% LC_ALL=ko_KR.UTF-8 python Python 2.4 (#2, Feb 4 2005, 12:07:54) [GCC 3.4.2 [FreeBSD] 20040728] on freebsd5 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.getlocale() (None, None) >>> locale.setlocale(locale.LC_ALL, '') 'ko_KR.UTF-8' >>> locale.getlocale() ['ko_KR', 'utf'] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177674&group_id=5470 From noreply at sourceforge.net Wed Apr 6 16:30:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 16:30:45 2005 Subject: [ python-Bugs-1177811 ] Exec Inside A Function Message-ID: Bugs item #1177811, was opened at 2005-04-06 14:30 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177811&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Andrew Wilkinson (andrew_j_w) Assigned to: Nobody/Anonymous (nobody) Summary: Exec Inside A Function Initial Comment: When 'exec'ing code that creates a function inside a function the defined function (fact in the example below) is created with the module level namespace as it's parent scope. The following should return 2 however it raises a NameError as fact is not defined. def f(): ? ? exec """ def fact(x): ? ? if x==1: ? ? ? ? return 1 ? ? else: ? ? ? ? return x*fact(x-1) """ ? ? return fact f()(2) If you run following code... def f(): ? ? exec """ def fact(x): ? ? if x==1: ? ? ? ? return 1 ? ? else: ? ? ? ? return x*fact(x-1) """ in locals() ? ? return fact ... it works as expected. The documentation states that "In all cases, if the optional parts are omitted, the code is executed in the current scope." That is clearly not the case here as the 'fact' function is set with the module level scope as it's parent scope. It would appear to me that either this a documentation bug or a flaw in exec. I sincerely hope this a bug in exec and not the desired behaviour as it doesn't make any sense to me... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177811&group_id=5470 From noreply at sourceforge.net Wed Apr 6 17:06:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 17:07:01 2005 Subject: [ python-Bugs-1177831 ] (?(id)yes|no) only works when referencing the first group Message-ID: Bugs item #1177831, was opened at 2005-04-06 17:06 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177831&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Andr? Malo (ndparker) Assigned to: Nobody/Anonymous (nobody) Summary: (?(id)yes|no) only works when referencing the first group Initial Comment: (?(id)yes|no) only works when referencing the first group Referencing other marked groups may lead to weird results. The problem is, that the compiler stores the following code: ... (op = GROUPREF_EXISTS) while the matcher expects: ... where group is and is . This is the problematic code in sre_compile.py (1.57): 168 elif op is GROUPREF_EXISTS: 169 emit(OPCODES[op]) 170 emit((av[0]-1)*2) 171 skipyes = _len(code); emit(0) 172 _compile(code, av[1], flags) changing line 170 to emit(av[0]-1) fixes the bug. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177831&group_id=5470 From noreply at sourceforge.net Wed Apr 6 19:55:44 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 19:55:47 2005 Subject: [ python-Bugs-1177964 ] Iterator on Fileobject gives no MemoryError Message-ID: Bugs item #1177964, was opened at 2005-04-06 19:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177964&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Folke Lemaitre (zypher) Assigned to: Nobody/Anonymous (nobody) Summary: Iterator on Fileobject gives no MemoryError Initial Comment: The following problem has only been tested on linux. Suppose at a certain time that your machine can allocate a maximum of X megabytes of memory. Allocating more than X should result in python MemoryErrors. Also suppose you have a file containing one big line taking more than X bytes (Large binary file for example). In this case, if you read lines from a file through the file objects iterator, you do NOT get the expected MemoryError as result, but an empty list. To reproduce, create a file twice as big as your machines memory and disable the swap. If you run the following code: # import os.path # # def test(input): # print "Testing %s (%sMB)"%(repr(input), os.path.getsize(input)/(1024.0*1024.0)) # count = 0 # for line in open(input): # count = count + 1 # print " >> Total Number of Lines: %s"%count # # if __name__ == "__main__": # test('test.small') # test('test.big') you'll get something like: # folke@wladimir devel $ python2.4 bug.py # Testing 'test.small' (20.0MB) # >> Total Number of Lines: 1 # Testing 'test.big' (2000.0MB) # >> Total Number of Lines: 0 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177964&group_id=5470 From noreply at sourceforge.net Wed Apr 6 20:52:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 20:52:55 2005 Subject: [ python-Feature Requests-1177998 ] Add a settimeout to ftplib.FTP object Message-ID: Feature Requests item #1177998, was opened at 2005-04-06 20:52 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1177998&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Juan Antonio Vali?o Garc?a (juanval) Assigned to: Nobody/Anonymous (nobody) Summary: Add a settimeout to ftplib.FTP object Initial Comment: It will be usefull that the FTP object of ftplib had a settimeout method to setup a timeout for the connection, because the only way of doing this is to use the socket.setdefaulttimeout method, and this is very dangerous when you are using threads. Thanks and keep up the work ! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1177998&group_id=5470 From noreply at sourceforge.net Wed Apr 6 20:53:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 20:53:21 2005 Subject: [ python-Feature Requests-1177998 ] Add a settimeout to ftplib.FTP object Message-ID: Feature Requests item #1177998, was opened at 2005-04-06 20:52 Message generated for change (Settings changed) made by juanval You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1177998&group_id=5470 >Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Juan Antonio Vali?o Garc?a (juanval) Assigned to: Nobody/Anonymous (nobody) Summary: Add a settimeout to ftplib.FTP object Initial Comment: It will be usefull that the FTP object of ftplib had a settimeout method to setup a timeout for the connection, because the only way of doing this is to use the socket.setdefaulttimeout method, and this is very dangerous when you are using threads. Thanks and keep up the work ! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1177998&group_id=5470 From noreply at sourceforge.net Wed Apr 6 22:37:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 6 22:37:50 2005 Subject: [ python-Bugs-1161595 ] Minor error in section 3.2 Message-ID: Bugs item #1161595, was opened at 2005-03-11 19:29 Message generated for change (Settings changed) made by jyby You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1161595&group_id=5470 Category: Documentation Group: None >Status: Deleted Resolution: Invalid Priority: 1 Submitted By: Jeremy Barbay (jyby) Assigned to: Nobody/Anonymous (nobody) Summary: Minor error in section 3.2 Initial Comment: In the section "3.2 First Steps Towards Programming " of the Python tutorial (http://docs.python.org/tut/node5.html), the output of both implementations of the Fibonacci sequence computation is incorrect. As written, only one 1 should be output. You should either remove one 1 from the input, or replace the lines "print b" and "print b," by "print a" and "print a,". This is minor but might confuse unnecessarily beginners. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-06 02:52 Message: Logged In: YES user_id=971153 It indeed seems that the output in tutorial is correct could you close or delete the bug then? Thanks ---------------------------------------------------------------------- Comment By: Jeremy Barbay (jyby) Date: 2005-03-11 19:40 Message: Logged In: YES user_id=149696 All my apologies: I didn't check my code correctly: as the algorithm is initializing a with 0 instead of 1, the output is correct. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1161595&group_id=5470 From noreply at sourceforge.net Thu Apr 7 00:45:08 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 00:45:17 2005 Subject: [ python-Bugs-1178136 ] cgitb.py support for frozen images Message-ID: Bugs item #1178136, was opened at 2005-04-06 23:45 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178136&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Barry Alan Scott (barry-scott) Assigned to: Nobody/Anonymous (nobody) Summary: cgitb.py support for frozen images Initial Comment: cgitb.py does not report the line numbers in the stack trace if the python program is frozen. This is because the source code is not available. The attached patch fixes this problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178136&group_id=5470 From noreply at sourceforge.net Thu Apr 7 00:48:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 00:48:42 2005 Subject: [ python-Bugs-1178141 ] urllib.py overwrite HTTPError code with 200 Message-ID: Bugs item #1178141, was opened at 2005-04-06 23:48 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178141&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Barry Alan Scott (barry-scott) Assigned to: Nobody/Anonymous (nobody) Summary: urllib.py overwrite HTTPError code with 200 Initial Comment: I found this bug while trying to understand why a 404 Not Found error was reported as a 200 Not Found. Turns out the HTTPError's self.code is overwritten with 200 after the 404 was correctly assigned. The attached patch fixes the problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178141&group_id=5470 From noreply at sourceforge.net Thu Apr 7 00:52:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 00:52:50 2005 Subject: [ python-Bugs-1178145 ] urllib2.py assumes 206 is an error Message-ID: Bugs item #1178145, was opened at 2005-04-06 23:52 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178145&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Barry Alan Scott (barry-scott) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2.py assumes 206 is an error Initial Comment: I'm writting code that uses the Range header. The correct response is 206, but the urllib2.py is coded to treat any code that is not 200 as error. The correct code needs to treat 200 to 299 as success. The attached patch fixes the problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178145&group_id=5470 From noreply at sourceforge.net Thu Apr 7 01:04:25 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 01:04:28 2005 Subject: [ python-Bugs-1178148 ] cgitb.py report wrong line number Message-ID: Bugs item #1178148, was opened at 2005-04-07 00:04 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178148&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Barry Alan Scott (barry-scott) Assigned to: Nobody/Anonymous (nobody) Summary: cgitb.py report wrong line number Initial Comment: Given code like try: raise 'bug' except ValueError: pass # cgitb.py think 'bug' is here cgitb.py will report that the exception 'bug' is at the pass line. This is a time waster until you figure that this problem exists. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178148&group_id=5470 From noreply at sourceforge.net Thu Apr 7 06:09:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 06:09:05 2005 Subject: [ python-Bugs-1178255 ] 256 should read 255 in operator module docs Message-ID: Bugs item #1178255, was opened at 2005-04-06 21:09 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178255&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Dan Everhart (dieseldann) Assigned to: Nobody/Anonymous (nobody) Summary: 256 should read 255 in operator module docs Initial Comment: In section 3.10 of the Python Library Reference, in the text near the bottom of the page which reads: Example: Build a dictionary that maps the ordinals from 0 to 256 to their character equivalents. the 256 should be replaced with 255, to match the code given. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178255&group_id=5470 From noreply at sourceforge.net Thu Apr 7 06:40:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 06:40:06 2005 Subject: [ python-Bugs-1178255 ] 256 should read 255 in operator module docs Message-ID: Bugs item #1178255, was opened at 2005-04-06 23:09 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178255&group_id=5470 Category: Documentation Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Dan Everhart (dieseldann) Assigned to: Nobody/Anonymous (nobody) Summary: 256 should read 255 in operator module docs Initial Comment: In section 3.10 of the Python Library Reference, in the text near the bottom of the page which reads: Example: Build a dictionary that maps the ordinals from 0 to 256 to their character equivalents. the 256 should be replaced with 255, to match the code given. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-06 23:40 Message: Logged In: YES user_id=80475 Fixed. Thanks for the report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178255&group_id=5470 From noreply at sourceforge.net Thu Apr 7 06:54:11 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 06:54:13 2005 Subject: [ python-Bugs-1178269 ] operator.isMappingType and isSequenceType on instances Message-ID: Bugs item #1178269, was opened at 2005-04-06 21:54 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178269&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Dan Everhart (dieseldann) Assigned to: Nobody/Anonymous (nobody) Summary: operator.isMappingType and isSequenceType on instances Initial Comment: Python Library Reference section 3.10 (module operator) claims that IsMappingType() and isSequenceType() return true for instance objects. Yet: ActivePython 2.4 Build 243 (ActiveState Corp.) based on Python 2.4 (#60, Nov 30 2004, 09:34:21) [MSC v.1310 32 bit (Intel)] Type "help", "copyright", "credits" or "license" for more informatio >>> class c: pass ... >>> x = c() >>> from operator import * >>> isMappingType(x) False >>> isSequenceType(x) False >>> x <__main__.c instance at 0x009EA7B0> ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178269&group_id=5470 From noreply at sourceforge.net Thu Apr 7 07:38:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 07:38:56 2005 Subject: [ python-Bugs-1178269 ] operator.isMappingType and isSequenceType on instances Message-ID: Bugs item #1178269, was opened at 2005-04-06 23:54 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178269&group_id=5470 Category: Documentation Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Dan Everhart (dieseldann) Assigned to: Nobody/Anonymous (nobody) Summary: operator.isMappingType and isSequenceType on instances Initial Comment: Python Library Reference section 3.10 (module operator) claims that IsMappingType() and isSequenceType() return true for instance objects. Yet: ActivePython 2.4 Build 243 (ActiveState Corp.) based on Python 2.4 (#60, Nov 30 2004, 09:34:21) [MSC v.1310 32 bit (Intel)] Type "help", "copyright", "credits" or "license" for more informatio >>> class c: pass ... >>> x = c() >>> from operator import * >>> isMappingType(x) False >>> isSequenceType(x) False >>> x <__main__.c instance at 0x009EA7B0> ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178269&group_id=5470 From noreply at sourceforge.net Thu Apr 7 10:54:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 10:54:06 2005 Subject: [ python-Bugs-1175967 ] StringIO and cStringIO don't provide 'name' attribute Message-ID: Bugs item #1175967, was opened at 2005-04-03 22:20 Message generated for change (Comment added) made by jvr You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175967&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: logistix (logistix) Assigned to: Nobody/Anonymous (nobody) Summary: StringIO and cStringIO don't provide 'name' attribute Initial Comment: Documentation explicitly states that file-like objects should return a repr-style psuedoname. Patch is attached. ---------------------------------------------------------------------- >Comment By: Just van Rossum (jvr) Date: 2005-04-07 10:54 Message: Logged In: YES user_id=92689 The documentation also says "This is a read-only attribute and may not be present on all file-like objects.", so I'm inclined to close as "won't fix". I'm sure many in-the-wild file-like objects don't support it, either, so depending on its existence is bad style at best. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175967&group_id=5470 From noreply at sourceforge.net Thu Apr 7 10:55:21 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 10:55:23 2005 Subject: [ python-Bugs-660083 ] GNU readline 4.2 prompt issue Message-ID: Bugs item #660083, was opened at 2002-12-30 17:47 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660083&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: GNU readline 4.2 prompt issue Initial Comment: When Python is linked with GNU readline 4.2, using raw_input() without a prompt causes the most recent non-empty prompt to be used. GNU readline 4.1 and before don't have this behavior. I don't want it, but I don't know how to disable it (I am passing an empty string for the prompt to readline()). ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-07 09:55 Message: Logged In: YES user_id=6656 I can't reproduce this. (looking at old open readline bugs -- me?) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660083&group_id=5470 From noreply at sourceforge.net Thu Apr 7 11:03:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 11:03:22 2005 Subject: [ python-Bugs-971213 ] another threads+readline+signals nasty Message-ID: Bugs item #971213, was opened at 2004-06-11 16:30 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=971213&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Nobody/Anonymous (nobody) Summary: another threads+readline+signals nasty Initial Comment: python -c "import time, readline, thread; thread.start_new_thread(raw_input, ()); time.sleep(2)" Segfaults on ^C Fails on Linux, freebsd. On linux (FC - using kernel 2.6.1, glibc 2.3.3, gcc-3.3.3) (gdb) where #0 0x002627a2 in _dl_sysinfo_int80 () from /lib/ld-linux.so.2 #1 0x008172b1 in ___newselect_nocancel () from /lib/tls/libc.so.6 #2 0x0011280b in time_sleep (self=0x0, args=0xb7fe17ac) at Modules/timemodule.c:815 on FreeBSD 5.2.1-RC, a different error. Fatal error 'longjmp()ing between thread contexts is undefined by POSIX 1003.1' at line 72 in file /usr/src/lib/libc_r/uthread/uthread_jmp.c (errno = 2) Abort (core dumped) ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-07 10:03 Message: Logged In: YES user_id=6656 I think this is fixed now, as in I can't reproduce it with CVS HEAD. Not sure why! I can think of a few fixes that might be responsible. It still messes the terminal up, though. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-08-07 22:41 Message: Logged In: YES user_id=6656 Bah, this still segfaults with CVS head. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-06-21 11:45 Message: Logged In: YES user_id=6656 Can you try the patch that's *now* in 960406? It seems to help for me (but I really would rather not think too hard about this!). ---------------------------------------------------------------------- Comment By: Michal Pasternak (mpasternak) Date: 2004-06-11 16:43 Message: Logged In: YES user_id=799039 readline used on FreeBSD was readline-4.3pl5; everything else: gcc 3.3.3, ncurses, libc were standard from 5.2.1. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-06-11 16:39 Message: Logged In: YES user_id=29957 The patch in #960406 doesn't help here. The FC test system also has readline-4.3, if it helps, as does the FreeBSD box. It apparently doesn't crash on OSX. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-06-11 16:38 Message: Logged In: YES user_id=6656 Hmm. Doesn't crash on OS X. Messes the terminal up good and proper, though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=971213&group_id=5470 From noreply at sourceforge.net Thu Apr 7 11:07:00 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 11:07:06 2005 Subject: [ python-Bugs-1009263 ] non-ascii readline input crashes python Message-ID: Bugs item #1009263, was opened at 2004-08-14 19:19 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1009263&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: paul rubin (phr) Assigned to: Nobody/Anonymous (nobody) Summary: non-ascii readline input crashes python Initial Comment: Start Python 2.3.4 on Linux command line: $ python Python 2.3.4 (#1, Jul 24 2004, 19:45:58) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> Enter the string "2?3" (2 times 3, the symbol between 2 and 3 is a multiplication sign, I think Unicode c397 or maybe 97c3 if I have it byte swapped) and hit return. Python reports a syntax error. Hit Ctrl-P to edit the line with readline. Use ctrl-D to delete the first character of the input line, and Python instantly crashes with a seg fault. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-07 10:07 Message: Logged In: YES user_id=6656 Closing for want of response. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-09-07 17:02 Message: Logged In: YES user_id=6656 ping ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-08-16 13:29 Message: Logged In: YES user_id=6656 You should check the $LANG or $LC_ALL env vars. Or just run 'locale' in the shell. What version of readline are you using? Both instinct, and looking at the backtrace, suggest a readline bug to me. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-08-15 02:55 Message: Logged In: YES user_id=72053 Backtrace follows. I don't know how to tell my locale. My LOCALE shell variable is not set to anything--is there something else I need to check? Since the "2?3" displays properly in my terminal window, I think I'm likely using UTF-8. (gdb) r Starting program: /usr/local/bin/python [New Thread 1074951328 (LWP 6561)] Python 2.3.4 (#1, Jul 24 2004, 19:45:58) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> 2?3 File "", line 1 2?3 ^ SyntaxError: invalid syntax >>> 2?3 Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 1074951328 (LWP 6561)] 0x401eb687 in _rl_get_char_len () from /usr/lib/libreadline.so.4 (gdb) bt #0 0x401eb687 in _rl_get_char_len () from /usr/lib/libreadline.so.4 #1 0x401eb701 in _rl_compare_chars () from /usr/lib/libreadline.so.4 #2 0x401e0030 in rl_redisplay () from /usr/lib/libreadline.so.4 #3 0x401de816 in rl_redisplay () from /usr/lib/libreadline.so.4 #4 0x401d2588 in readline_internal_char () from /usr/lib/libreadline.so.4 #5 0x401d2695 in readline_internal_char () from /usr/lib/libreadline.so.4 #6 0x401d26ce in readline_internal_char () from /usr/lib/libreadline.so.4 #7 0x401d219f in readline () from /usr/lib/libreadline.so.4 #8 0x40019074 in call_readline (sys_stdin=0x4212fcc0, sys_stdout=0x4212fe20, prompt=0x40177254 ">>> ") at /home/phr/python/Python-2.3.4/Modules/readline.c:679 #9 0x080e6110 in PyOS_Readline (sys_stdin=0x4212fcc0, sys_stdout=0x4212fe20, prompt=0x40177254 ">>> ") at Parser/myreadline.c:183 #10 0x08056c47 in tok_nextc (tok=0x817f428) at Parser/tokenizer.c:683 #11 0x08055d8f in tok_get (tok=0x817f428, p_start=0xbfffecc4, p_end=0xbfffecc8) at Parser/tokenizer.c:1014 #12 0x080559de in PyTokenizer_Get (tok=0x817f428, p_start=0xbfffecc4, p_end=0xbfffecc8) at Parser/tokenizer.c:1414 #13 0x0805543a in parsetok (tok=0x817f428, g=0x8126b88, start=256, err_ret=0xbfffed10, flags=0) at Parser/parsetok.c:125 #14 0x080cb0e2 in PyRun_InteractiveOneFlags (fp=0x4212fcc0, filename=0x80f004a "", flags=0xbfffedf8) at Python/pythonrun.c:738 #15 0x080caf3b in PyRun_InteractiveLoopFlags (fp=0x4212fcc0, filename=0x80f004a "", flags=0xbfffedf8) at Python/pythonrun.c:690 #16 0x080cc2ee in PyRun_AnyFileExFlags (fp=0x4212fcc0, filename=0x80f004a "", closeit=0, flags=0xbfffedf8) at Python/pythonrun.c:653 #17 0x08054d32 in Py_Main (argc=0, argv=0xbfffee74) at Modules/main.c:415 #18 0x080548ab in main (argc=1, argv=0xbfffee74) at Modules/python.c:23 #19 0x420156a4 in __libc_start_main () from /lib/tls/libc.so.6 ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-08-14 22:10 Message: Logged In: YES user_id=55188 What locale do you use? I couldn't reproduce the problem on any of utf-8, iso8859-1, C, euc-kr locale. And, can you please attach a backtrace for the segfault point? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1009263&group_id=5470 From noreply at sourceforge.net Thu Apr 7 11:07:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 11:07:46 2005 Subject: [ python-Bugs-1103023 ] raw_input problem with readline and UTF8 Message-ID: Bugs item #1103023, was opened at 2005-01-15 18:10 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1103023&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Casey Crabb (airog) Assigned to: Nobody/Anonymous (nobody) Summary: raw_input problem with readline and UTF8 Initial Comment: Backspace doesn't remove all bytes of a multi-byte UTF-8 character. To reproduce the problem: $ export LANG=en_US.UTF-8 $ python Python 2.3.4 (#1, Jun 11 2004, 16:35:29) [GCC 3.3.3 20040412 (Gentoo Linux 3.3.3-r3, ssp-3.3-7, pie-8.5.3)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline >>> raw_input() # ?, return ? '\xc3\xa4' >>> raw_input() # ?, backspace, return '\xc3' >>> A small C program does not have the same problem: #include #include #include #include void pprint(const char *s); int main(void) { char *line; for (;;) { line = readline("> "); if (!line) break; pprint(line); free(line); } return 0; } void pprint(const char *s) { while (*s) { if (isprint(*s)) putchar(*s); else printf("\x%x", *s & 0xff); s++; } putchar('\n'); } ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-07 10:07 Message: Logged In: YES user_id=6656 Is this still relavent? ---------------------------------------------------------------------- Comment By: Matthew Mueller (donut) Date: 2005-02-07 07:36 Message: Logged In: YES user_id=65253 Hi, it looks like this might be the same problem already fixed in "[ 914291 ] Fix readline for utf-8 locales", but your python version is from before the fix went in. Can you try again with python 2.4, or 2.3.5 (when it is released, or the release23-maint branch of cvs)? Also, you could check if python's readline.so is linked to an older version of libreadline. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1103023&group_id=5470 From noreply at sourceforge.net Thu Apr 7 11:12:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 11:12:43 2005 Subject: [ python-Bugs-811844 ] readline+no threads = core dump Message-ID: Bugs item #811844, was opened at 2003-09-24 16:46 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=811844&group_id=5470 Category: Threads Group: Python 2.3 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: TEd Drain (teddrain) Assigned to: Nobody/Anonymous (nobody) Summary: readline+no threads = core dump Initial Comment: Build 2.3 or 2.3.1 on linux with the following configure command: ./configure --enable-shared --without-threads Try running the rlcompleter example. If you hit TAB with completions enabled, you'll get a core dump after you hit return on that line. Works fine if I build with threads enabled. This is a major problem for me since we have to deliver our software on machines that do not support pthreads (hpux 10.2 for example). ~: uname -a Linux XXXX 2.4.9-21 #1 Thu Jan 17 14:16:30 EST 2002 i686 unknown ~/builds-gcc2: python Python 2.3.1 (#1, Sep 24 2003, 08:13:26) [GCC 2.95.3 20010315 (release)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline, rlcompleter >>> readline.parse_and_bind("tab: complete") >>> readline.par<TAB> se_and_bind() Fatal Python error: PyThreadState_Get: no current thread Abort (core dumped) ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-07 10:12 Message: Logged In: YES user_id=6656 This seems to have been fixed, at some indeterminate time in the past. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2003-09-26 18:30 Message: Logged In: YES user_id=29957 Assuming the problem is just a shallow one, would the following patch "fix" the problem? It makes the crashes go away for me. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2003-09-24 17:17 Message: Logged In: YES user_id=6656 i see this too. oops. hopefully easy to fix... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=811844&group_id=5470 From noreply at sourceforge.net Thu Apr 7 11:14:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 11:14:57 2005 Subject: [ python-Bugs-846817 ] control-c is being sent to child thread rather than main Message-ID: Bugs item #846817, was opened at 2003-11-21 19:46 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=846817&group_id=5470 Category: Threads Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: sheshi sankineni (sheshi) Assigned to: Nobody/Anonymous (nobody) Summary: control-c is being sent to child thread rather than main Initial Comment: Hi All, We just migrated from Python-2.2.1 to Python-2.3.1 and noticed a behavior change with respect to control-c + threads and importing readline module. We have developed a CLI (command line interface) to our hardware using Python (communicating with CORBA). I am providing you with the code (below) which demonstrates the changed behavior in Python-2.3.1 with "import readline" & without "import readline" on Linux Redhat 7.2. The same program works fine on Python- 2.2.1 with or without importing readline and doing control-c ALWAYS trapped by the main program. Any help on this issue will be appreciated thanks -sheshi cli.py ------- import os import signal import sys import cliEngine def handleKeyboardInterrupt(signalNumber, frame) : print 'MAIN THREAD Control-C' def handleSigUsr1(signalNumber, frame) : pass # The main function if __name__ == '__main__': try : # Register a signal handler for ctrl-C, control-z signal.signal(signal.SIGINT, handleKeyboardInterrupt) signal.signal(signal.SIGUSR1, handleSigUsr1) signal.signal(signal.SIGTSTP, signal.SIG_IGN) cliEngine.engine = cliEngine.CliEngine() cliEngine.engine.setPid(os.getpid()) cliEngine.engine.start() while cliEngine.engine.cliEngineRunning : signal.pause() print 'Exiting the main thread' finally : pass cliEngine.py ----------------- import os import signal import sys import threading import readline engine = None class CliEngine(threading.Thread) : # Static members of the class cliEngineRunning = 0 mainThreadPid = 0 def __init__(self) : threading.Thread.__init__(self) self.cliEngineRunning = 1 engine = self def setPid(self, mainPid) : self.mainThreadPid = mainPid def run(self) : print 'calling CliEngine run' self.cliEngineRunning = 1 self.runEngine() def runEngine(self) : while self.cliEngineRunning : try : line = raw_input('# ') if line == "logout" : self.cliEngineRunning = 0 except KeyboardInterrupt : print 'CHILD THREAD Control-C (KeyboardInterrupt)' except EOFError : # ctrl-d to logout from CLI self.cliEngineRunning = 0 except : print 'Unknown Exception' self.cliEngineRunning = 0 print 'Returning from cliEngine.runEngine()' If I remove "import readline" in the above file, control-c is always trapped by the main thread (cli.py) as shown by the output below >>python cli.py calling CliEngine run # MAIN THREAD Control-C # MAIN THREAD Control-C # logout Returning from cliEngine.runEngine() Exiting the main thread If in the above Python file (cliEngine.py), If I include "import readline", control-c will be trapped by the child thread (CliEngine) instead of the main thread, as shown by the output below and the program terminates with segmentation fault >>python cli.py calling CliEngine run # CHILD THREAD Control-C (KeyboardInterrupt) # CHILD THREAD Control-C (KeyboardInterrupt) # # logout Returning from cliEngine.runEngine() Segmentation fault ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-07 10:14 Message: Logged In: YES user_id=6656 Is this still an issue with Python 2.4? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=846817&group_id=5470 From noreply at sourceforge.net Thu Apr 7 11:26:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 11:26:27 2005 Subject: [ python-Bugs-1054041 ] Python doesn't exit with proper resultcode on SIGINT Message-ID: Bugs item #1054041, was opened at 2004-10-25 21:27 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1054041&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: James Y Knight (foom) Assigned to: Nobody/Anonymous (nobody) Summary: Python doesn't exit with proper resultcode on SIGINT Initial Comment: If you kill a python process with SIGINT (e.g. control-c), it catches the signal, and raises a KeyboardInterrupt. If the KeyboardInterrupt propagates to the top level, python exits. However, it exits with a result of 1, not a result of SIGINT. This means that if you run python in a shell script, the script will not properly exit on C-c. When exiting because of a KeyboardInterrupt, python ought to reraise the SIGINT, as follows, so that the exit code is correct for the caller: signal(SIGINT, SIG_DFL); kill(getpid(), SIGINT); See also http://www.cons.org/cracauer/sigint.html for a more detailed discussion on the topic. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-07 10:26 Message: Logged In: YES user_id=6656 Feel like writing a patch? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1054041&group_id=5470 From noreply at sourceforge.net Thu Apr 7 11:51:28 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 11:51:30 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 10:05 Message generated for change (Comment added) made by vmlinuxz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Carroll (vmlinuxz) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- >Comment By: David Carroll (vmlinuxz) Date: 2005-04-07 09:51 Message: Logged In: YES user_id=684143 Thank you for leaving this open. I will try and get your example today. I'm too uneducated to realize what I'm doing wrong, so maybe you can use my ignorance for the common good =). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-05 04:31 Message: Logged In: YES user_id=80475 I fixed the IndexError. It should have been a protocol error. See Lib/ftplib.py 1.74. Leaving the bug open so the OP can make a more detailed post so we can see what the root issue is about. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 03:37 Message: Logged In: YES user_id=971153 It turns out that the code in plain text bug report is more or less readable, so I can see what you were trying to do... Seems like you are doing ftp.dir("LIST "+ftpDir) But ftplib docs: http://www.python.org/doc/current/lib/ftp-objects.html say that you need to specify the directory ftp.dir(ftpDir) so it seems that you are using ftp.dir() incorrectly.... But, even then IndexError seems like a wrong kind of exception for this kind of error... So, I guess, my original request for a working code snippet which would allow to reproduce the bug still stands.. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 02:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Thu Apr 7 14:33:39 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 14:33:42 2005 Subject: [ python-Bugs-1178484 ] Erroneous line number error in Py2.4.1 Message-ID: Bugs item #1178484, was opened at 2005-04-07 15:33 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178484&group_id=5470 Category: Parser/Compiler Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Timo Linna (tilinna) Assigned to: Nobody/Anonymous (nobody) Summary: Erroneous line number error in Py2.4.1 Initial Comment: For some reason Python 2.3.5 reports the error in the following program correctly: File "C:\Temp\problem.py", line 7 SyntaxError: unknown decode error ..whereas Python 2.4.1 reports an invalid line number: File "C:\Temp\problem.py", line 2 SyntaxError: unknown decode error ----- problem.py starts ----- # -*- coding: ascii -*- """ Foo bar """ # ? is not allowed in ascii coding ----- problem.py ends ----- Without the encoding declaration both Python versions report the usual deprecation warning (just like they should be doing). My environment: Windows 2000 + SP3. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178484&group_id=5470 From noreply at sourceforge.net Thu Apr 7 14:53:45 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 14:53:47 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Thu Apr 7 16:28:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 16:29:00 2005 Subject: [ python-Bugs-1178484 ] Erroneous line number error in Py2.4.1 Message-ID: Bugs item #1178484, was opened at 2005-04-07 14:33 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178484&group_id=5470 Category: Parser/Compiler Group: Python 2.4 Status: Open >Resolution: Accepted Priority: 5 Submitted By: Timo Linna (tilinna) Assigned to: Nobody/Anonymous (nobody) Summary: Erroneous line number error in Py2.4.1 Initial Comment: For some reason Python 2.3.5 reports the error in the following program correctly: File "C:\Temp\problem.py", line 7 SyntaxError: unknown decode error ..whereas Python 2.4.1 reports an invalid line number: File "C:\Temp\problem.py", line 2 SyntaxError: unknown decode error ----- problem.py starts ----- # -*- coding: ascii -*- """ Foo bar """ # ? is not allowed in ascii coding ----- problem.py ends ----- Without the encoding declaration both Python versions report the usual deprecation warning (just like they should be doing). My environment: Windows 2000 + SP3. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 16:28 Message: Logged In: YES user_id=89016 The reason for this is the new codec buffering code in 2.4: The codec might read and decode more data from the byte stream than is neccessary for decoding one line. I.e. when reading line n, the codec might decode bytes that belong to line n+1, n+2 etc. too. If there's a decoding error in this data, line n gets reported. I don't think there's a simple fix for this. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178484&group_id=5470 From noreply at sourceforge.net Thu Apr 7 16:33:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 16:33:23 2005 Subject: [ python-Bugs-1174606 ] Reading /dev/zero causes SystemError Message-ID: Bugs item #1174606, was opened at 2005-03-31 22:48 Message generated for change (Comment added) made by jepler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Reading /dev/zero causes SystemError Initial Comment: $ python -c 'open("/dev/zero").read()' Traceback (most recent call last): File "", line 1, in ? SystemError: ../Objects/stringobject.c:3316: bad argument to internal function Compare with this two variants: $ python -c 'open("/dev/zero").read(2**31-1)' Traceback (most recent call last): File "", line 1, in ? MemoryError $ python -c 'open("/dev/zero").read(2**31)' Traceback (most recent call last): File "", line 1, in ? OverflowError: long int too large to convert to int The unsized read should produce either MemoryError or OverflowError instead of SystemError. Tested with Python 2.2, 2.3, and 2.4. ---------------------------------------------------------------------- Comment By: Jeff Epler (jepler) Date: 2005-04-07 09:33 Message: Logged In: YES user_id=2772 To my surprise, the st_size field *is* undefined for devices, according to http://www.opengroup.org/onlinepubs/009695399/basedefs/sys/stat.h.html My Linux consistently gives 0 for device nodes, but I guess Unix might not. jafo, loweis: read() from /dev/zero is special cased in linux. In 2.4.20-8, function read_zero_pagealined, a comment says "for private mappings, just map in zero pages", which will be very fast and not actually write any memory or blocks of swap. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 02:17 Message: Logged In: YES user_id=81797 I'm quite sure that the 2GB is not getting filled when doing this. After running the first command, and checking /proc/meminfo, I see that only 46MB is shown as free, which means that there was no more than this amount of RAM consumed. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 02:06 Message: Logged In: YES user_id=21627 The surprising aspect is that the memory *is* being used. Python allocates 2GB of memory, and then passes this to read(2) (through stdio) to fill it with the contents of /dev/zero. This should cause a write operation to the memory pages, which in turn should cause them to consume actual memory. For some reason, this doesn't happen. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 01:52 Message: Logged In: YES user_id=81797 Linux can do a very fast allocation if it has swap available. It reserves space, but does not actually assign the memory until you try to use it. In my case, I have 1GB of RAM, around 700MB free, and another 2GB in swap. So, I have plenty unless I use it. In C I can malloc 1GB and unless I write every page in that block the system doesn't really give the pages to the process. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 01:40 Message: Logged In: YES user_id=21627 The problem is different. Instead, _PyString_Resize complains that the new buffersize of the string is negative. This in turn happens because the string manages to get larger >2GB, which in turn happens because buffersize is size_t, yet _PyString_Resize expects int. I don't know how Linux manages to allocate such a large string without thrashing. There is a minor confusion with stat() as well: new_buffersize tries to find out how much bytes are left to the end of the file. In the case of /dev/zero, both fstat and lseek are "lying" by returning 0. As lseek returns 0, ftell is invoked and returns non-zero. Then, newbuffer does not trust the values, and just adds BIGCHUNK. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 22:39 Message: Logged In: YES user_id=81797 I am able to reproduce this on a Fedora Core 3 Linux system: >>> fp = open('/dev/zero', 'rb') >>> d = fp.read() Traceback (most recent call last): File "", line 1, in ? MemoryError >>> print os.stat('/dev/zero').st_size 0 What about only trusting st_size if the file is a regular file, not a directory or other type of special file? Sean ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 06:31 Message: Logged In: YES user_id=4771 os.stat() doesn't always give consistent results on dev files. On my machine for some reason os.stat('/dev/null') appears to be random (and extremely large). I suspect that on the OP's machine os.stat('/dev/zero') is not 0 either, but a random number that turns out to be negative, hence a "bad argument" SystemError. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 15:42 Message: Logged In: YES user_id=21627 I think it should trust the stat result, and then find that it cannot allocate that much memory. Actually, os.stat("/dev/zero").st_size is 0, so something else must be going on. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 03:58 Message: Logged In: YES user_id=4771 I think that file.read() with no argument needs to be more conservative. Currently it asks and trusts the stat() to get the file size, but this can lead to just plain wrong results on special devices. (I had the problem that open('/dev/null').read() would give a MemoryError!) We can argue whether plain read() on special devices is a good idea or not, but I guess that not blindly trusting stat() if it returns huge values could be a good idea. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1174606&group_id=5470 From noreply at sourceforge.net Thu Apr 7 17:15:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 17:15:33 2005 Subject: [ python-Bugs-1176893 ] Readline segfault Message-ID: Bugs item #1176893, was opened at 2005-04-05 04:50 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Walter D?rwald (doerwalter) >Assigned to: Mark Hammond (mhammond) Summary: Readline segfault Initial Comment: The latest change to the readline module has broken tab completion: ./python Python 2.5a0 (#1, Apr 5 2005, 01:14:33) [GCC 3.3.5 (Gentoo Linux 3.3.5-r1, ssp-3.3.2-3, pie-8.7.7.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline, rlcompleter [25913 refs] >>> readline.parse_and_bind("tab: complete") [25913 refs] >>> Segmentation fault [Press tab after the parse_and_bind() call] ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-07 11:15 Message: Logged In: YES user_id=31435 Sorry, can't make time for this. Reassigned to Mark in case he can. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-05 06:27 Message: Logged In: YES user_id=6656 Or maybe this one line patch is the answer (it certainly fixes this case). Tim, can you spare a minute to think about this? The patch simply adds a check to PyEval_ReleaseThread that doesn't call PyThread_release_lock if the GIL hasn't been allocated. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-05 06:18 Message: Logged In: YES user_id=6656 I'm going to go out on a limb and suggest this is a bug in the PyGilState_ functions. The problem burrows down to calling PyThread_release_lock(interpreter_lock) when interpreter_lock is NULL, i.e. PyEval_InitThreads hasn't been called. (if you start a thread before running the crashing code, it doesn't crash, because the GIL has been allocated). Not sure what the solution is, or who to bug (time to read cvs log, I guess). A silly workaround is to put PyEval_InitThreads in initreadline. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 From noreply at sourceforge.net Thu Apr 7 17:25:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 17:25:26 2005 Subject: [ python-Bugs-1178624 ] 2.4.1 breaks pyTTS Message-ID: Bugs item #1178624, was opened at 2005-04-07 09:25 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Dieter Deyke (deyke) Assigned to: Nobody/Anonymous (nobody) Summary: 2.4.1 breaks pyTTS Initial Comment: On my Windows XP PRO SP2 system, the combination of Python 2.4, pywin32-203.win32-py2.4, and pyTTS-3.0.win32-py2.4 did work. Updating to 2.4.1 broke pyTTS. Removing everythin, then installing 2.4.1, win32, and pyTTS resulted in the same failure. Removing everything again, and installing 2.4, win32, and pyTTS fixed it again. The problem is that with 2.4.1 pyTTS reported that "SAPI" was not supported. Looking deeper I saw that it got a syntax error reading a generated com interface file. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 From noreply at sourceforge.net Thu Apr 7 17:56:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 17:56:26 2005 Subject: [ python-Bugs-1178624 ] 2.4.1 breaks pyTTS Message-ID: Bugs item #1178624, was opened at 2005-04-07 17:25 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Dieter Deyke (deyke) Assigned to: Nobody/Anonymous (nobody) Summary: 2.4.1 breaks pyTTS Initial Comment: On my Windows XP PRO SP2 system, the combination of Python 2.4, pywin32-203.win32-py2.4, and pyTTS-3.0.win32-py2.4 did work. Updating to 2.4.1 broke pyTTS. Removing everythin, then installing 2.4.1, win32, and pyTTS resulted in the same failure. Removing everything again, and installing 2.4, win32, and pyTTS fixed it again. The problem is that with 2.4.1 pyTTS reported that "SAPI" was not supported. Looking deeper I saw that it got a syntax error reading a generated com interface file. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 17:56 Message: Logged In: YES user_id=89016 Can you post the generated inferface file? Does this file contain long lines? Does it use a PEP 263 encoding directive? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 From noreply at sourceforge.net Thu Apr 7 18:46:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 18:46:22 2005 Subject: [ python-Bugs-660083 ] GNU readline 4.2 prompt issue Message-ID: Bugs item #660083, was opened at 2002-12-30 12:47 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660083&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: GNU readline 4.2 prompt issue Initial Comment: When Python is linked with GNU readline 4.2, using raw_input() without a prompt causes the most recent non-empty prompt to be used. GNU readline 4.1 and before don't have this behavior. I don't want it, but I don't know how to disable it (I am passing an empty string for the prompt to readline()). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-07 12:46 Message: Logged In: YES user_id=6380 Me neither, but then I've got GNU readline 4.3 here, not 4.2, and I'm to lazy to try to reproduce it. Assuming that you actually tested with 4.2, just close it. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-07 04:55 Message: Logged In: YES user_id=6656 I can't reproduce this. (looking at old open readline bugs -- me?) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660083&group_id=5470 From noreply at sourceforge.net Thu Apr 7 18:49:26 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 18:49:29 2005 Subject: [ python-Bugs-1178624 ] 2.4.1 breaks pyTTS Message-ID: Bugs item #1178624, was opened at 2005-04-07 09:25 Message generated for change (Comment added) made by deyke You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Dieter Deyke (deyke) Assigned to: Nobody/Anonymous (nobody) Summary: 2.4.1 breaks pyTTS Initial Comment: On my Windows XP PRO SP2 system, the combination of Python 2.4, pywin32-203.win32-py2.4, and pyTTS-3.0.win32-py2.4 did work. Updating to 2.4.1 broke pyTTS. Removing everythin, then installing 2.4.1, win32, and pyTTS resulted in the same failure. Removing everything again, and installing 2.4, win32, and pyTTS fixed it again. The problem is that with 2.4.1 pyTTS reported that "SAPI" was not supported. Looking deeper I saw that it got a syntax error reading a generated com interface file. ---------------------------------------------------------------------- >Comment By: Dieter Deyke (deyke) Date: 2005-04-07 10:49 Message: Logged In: YES user_id=132928 I attached the generated interface file. Loading this file into python by itself does NOT generate an error, but going into the pyTTS dir, starting python, and typing "import sapi" shows: C:\Python24\Lib\site-packages\pyTTS>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sapi Traceback (most recent call last): File "", line 1, in ? File "C:\Python24\lib\site-packages\pyTTS\sapi.py", line 10, in ? sapi_mod = gencache.EnsureModule('{C866CA3A-32F7-11D2-9602-00C04F8EE628}', 0 , 5, 0) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 393, in EnsureModule module = GetModuleForTypelib(typelibCLSID, lcid, major, minor) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 258, in GetModuleForTypelib mod = _GetModule(modName) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 629, in _GetModule mod = __import__(mod_name) File "C:\Python24\lib\site-packages\win32com\gen_py\C866CA3A-32F7-11D2-9602-00 C04F8EE628x0x5x0.py", line 1192 """Add""" ^ SyntaxError: invalid syntax >>> ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 09:56 Message: Logged In: YES user_id=89016 Can you post the generated inferface file? Does this file contain long lines? Does it use a PEP 263 encoding directive? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 From noreply at sourceforge.net Thu Apr 7 22:33:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 22:33:35 2005 Subject: [ python-Bugs-1178624 ] 2.4.1 breaks pyTTS Message-ID: Bugs item #1178624, was opened at 2005-04-07 17:25 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Dieter Deyke (deyke) Assigned to: Nobody/Anonymous (nobody) Summary: 2.4.1 breaks pyTTS Initial Comment: On my Windows XP PRO SP2 system, the combination of Python 2.4, pywin32-203.win32-py2.4, and pyTTS-3.0.win32-py2.4 did work. Updating to 2.4.1 broke pyTTS. Removing everythin, then installing 2.4.1, win32, and pyTTS resulted in the same failure. Removing everything again, and installing 2.4, win32, and pyTTS fixed it again. The problem is that with 2.4.1 pyTTS reported that "SAPI" was not supported. Looking deeper I saw that it got a syntax error reading a generated com interface file. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 22:33 Message: Logged In: YES user_id=89016 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- Comment By: Dieter Deyke (deyke) Date: 2005-04-07 18:49 Message: Logged In: YES user_id=132928 I attached the generated interface file. Loading this file into python by itself does NOT generate an error, but going into the pyTTS dir, starting python, and typing "import sapi" shows: C:\Python24\Lib\site-packages\pyTTS>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sapi Traceback (most recent call last): File "", line 1, in ? File "C:\Python24\lib\site-packages\pyTTS\sapi.py", line 10, in ? sapi_mod = gencache.EnsureModule('{C866CA3A-32F7-11D2-9602-00C04F8EE628}', 0 , 5, 0) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 393, in EnsureModule module = GetModuleForTypelib(typelibCLSID, lcid, major, minor) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 258, in GetModuleForTypelib mod = _GetModule(modName) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 629, in _GetModule mod = __import__(mod_name) File "C:\Python24\lib\site-packages\win32com\gen_py\C866CA3A-32F7-11D2-9602-00 C04F8EE628x0x5x0.py", line 1192 """Add""" ^ SyntaxError: invalid syntax >>> ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 17:56 Message: Logged In: YES user_id=89016 Can you post the generated inferface file? Does this file contain long lines? Does it use a PEP 263 encoding directive? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 From noreply at sourceforge.net Thu Apr 7 23:01:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 23:01:55 2005 Subject: [ python-Bugs-1178863 ] Variable.__init__ uses self.set(), blocking specialization Message-ID: Bugs item #1178863, was opened at 2005-04-07 23:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178863&group_id=5470 Category: Tkinter Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Emil (droun) Assigned to: Martin v. L?wis (loewis) Summary: Variable.__init__ uses self.set(), blocking specialization Initial Comment: in class Variable the constructor uses self.set() to set the value of the variable. This makes it hard and in some cases impossible to make a speciallization of Variable that performs some operation after changing the value. It would be preferable to use Tkinter.Variable.set(self, self._default) example: class ViewChanger(Tkinter.StringVar) : def __init__(self, views) : self.views = views Tkinter.StringVar.__init__(self) def set(self, newview) : self.views.activate(newview) Tkinter.StringVar.set(newview) get() and everything else will use the normal StringVar ipementation, but when the value is changed, a new view is activated. At creation here the self.views varaible will be accessed, method activate() will be called, and it might not even be fully initiallized yet. This is the normal case if you want to supply a variable to a widget, where a variable change may update the widget. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178863&group_id=5470 From noreply at sourceforge.net Thu Apr 7 23:20:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 7 23:21:01 2005 Subject: [ python-Bugs-1178872 ] Variable.__init__ uses self.set(), blocking specialization Message-ID: Bugs item #1178872, was opened at 2005-04-07 23:20 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178872&group_id=5470 Category: Tkinter Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Emil (droun) Assigned to: Martin v. L?wis (loewis) Summary: Variable.__init__ uses self.set(), blocking specialization Initial Comment: in class Variable the constructor uses self.set() to set the value of the variable. This makes it hard and in some cases impossible to make a speciallization of Variable that performs some operation after changing the value. It would be preferable to use Tkinter.Variable.set(self, self._default) example: class ViewChanger(Tkinter.StringVar) : def __init__(self, views) : self.views = views Tkinter.StringVar.__init__(self) def set(self, newview) : self.views.activate(newview) Tkinter.StringVar.set(newview) get() and everything else will use the normal StringVar ipementation, but when the value is changed, a new view is activated. At creation here the self.views varaible will be accessed, method activate() will be called, and it might not even be fully initiallized yet. This is the normal case if you want to supply a variable to a widget, where a variable change may update the widget. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178872&group_id=5470 From noreply at sourceforge.net Fri Apr 8 00:50:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 00:50:53 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Fri Apr 8 05:21:33 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 05:21:37 2005 Subject: [ python-Bugs-1176012 ] compiler module didn't get updated for "class foo():pass" Message-ID: Bugs item #1176012, was opened at 2005-04-03 16:21 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: logistix (logistix) >Assigned to: Brett Cannon (bcannon) Summary: compiler module didn't get updated for "class foo():pass" Initial Comment: Patch with fix and testcase is attached. ---------------------------------------------------------------------- Comment By: logistix (logistix) Date: 2005-04-03 17:01 Message: Logged In: YES user_id=699438 The parser module didn't get corrected either. Here is another patch with test case for that. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 From noreply at sourceforge.net Fri Apr 8 05:21:54 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 05:21:56 2005 Subject: [ python-Bugs-1176012 ] compiler module didn't get updated for "class foo():pass" Message-ID: Bugs item #1176012, was opened at 2005-04-03 16:21 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 Category: Python Library >Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: logistix (logistix) Assigned to: Brett Cannon (bcannon) Summary: compiler module didn't get updated for "class foo():pass" Initial Comment: Patch with fix and testcase is attached. ---------------------------------------------------------------------- Comment By: logistix (logistix) Date: 2005-04-03 17:01 Message: Logged In: YES user_id=699438 The parser module didn't get corrected either. Here is another patch with test case for that. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 From noreply at sourceforge.net Fri Apr 8 09:51:51 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 09:51:54 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by zosh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: zosh (zosh) Date: 2005-04-08 09:51 Message: Logged In: YES user_id=1254741 My test program that compiled correctly with "cc -c -g": #include int main () { void *p = setgroups; return 0; } The definition of setgroups in : #if _SGIAPI ... #if defined(_BSD_COMPAT) extern int setgroups(int, int *); #else /* !_BSD_COMPAT */ extern int setgroups(int, const gid_t *); #endif /* _BSD_COMPAT */ ... #endif /* _SGIAPI */ So it should catch one of these defines. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Fri Apr 8 10:06:57 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 10:07:00 2005 Subject: [ python-Bugs-660083 ] GNU readline 4.2 prompt issue Message-ID: Bugs item #660083, was opened at 2002-12-30 17:47 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660083&group_id=5470 Category: Extension Modules >Group: 3rd Party >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: GNU readline 4.2 prompt issue Initial Comment: When Python is linked with GNU readline 4.2, using raw_input() without a prompt causes the most recent non-empty prompt to be used. GNU readline 4.1 and before don't have this behavior. I don't want it, but I don't know how to disable it (I am passing an empty string for the prompt to readline()). ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-08 09:06 Message: Logged In: YES user_id=6656 Oops, I have readline 4.3 too. Still, I'm going to close this as out of date/3rd party. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-07 17:46 Message: Logged In: YES user_id=6380 Me neither, but then I've got GNU readline 4.3 here, not 4.2, and I'm to lazy to try to reproduce it. Assuming that you actually tested with 4.2, just close it. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-07 09:55 Message: Logged In: YES user_id=6656 I can't reproduce this. (looking at old open readline bugs -- me?) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=660083&group_id=5470 From noreply at sourceforge.net Fri Apr 8 12:13:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 12:13:42 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 10:05 Message generated for change (Comment added) made by vmlinuxz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Carroll (vmlinuxz) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- >Comment By: David Carroll (vmlinuxz) Date: 2005-04-08 10:13 Message: Logged In: YES user_id=684143 I achieve the same results with the following code. ftp = FTP('ftp.cdrom.com') ftp.login('anonymous', 'anonymous@') ftp.retrlines('LIST') This code works just fine with your patched ftplib.py, or the old ftplib.py against all public internet ftp servers I have tried it on. It works on the server at work only sporadically,, and raising the protocol exception the rest of the time. I'm inclined to think that there may be a problem with the protocol implementation on the server side. The server software version is wu-2.6.2(1), and the only oddity I see with it is a pretty sizeable delay before giving the login. Up to 10 seconds. I'd love to get a pointer of where to take this debugging process next. Perhaps I need to use a sniffer on the connection or something. I would love to see what ftplib.py is getting back that is breaking it. ---------------------------------------------------------------------- Comment By: David Carroll (vmlinuxz) Date: 2005-04-07 09:51 Message: Logged In: YES user_id=684143 Thank you for leaving this open. I will try and get your example today. I'm too uneducated to realize what I'm doing wrong, so maybe you can use my ignorance for the common good =). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-05 04:31 Message: Logged In: YES user_id=80475 I fixed the IndexError. It should have been a protocol error. See Lib/ftplib.py 1.74. Leaving the bug open so the OP can make a more detailed post so we can see what the root issue is about. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 03:37 Message: Logged In: YES user_id=971153 It turns out that the code in plain text bug report is more or less readable, so I can see what you were trying to do... Seems like you are doing ftp.dir("LIST "+ftpDir) But ftplib docs: http://www.python.org/doc/current/lib/ftp-objects.html say that you need to specify the directory ftp.dir(ftpDir) so it seems that you are using ftp.dir() incorrectly.... But, even then IndexError seems like a wrong kind of exception for this kind of error... So, I guess, my original request for a working code snippet which would allow to reproduce the bug still stands.. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 02:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Fri Apr 8 12:18:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 12:18:46 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 10:05 Message generated for change (Settings changed) made by vmlinuxz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open >Resolution: Accepted Priority: 5 Submitted By: David Carroll (vmlinuxz) >Assigned to: Raymond Hettinger (rhettinger) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- Comment By: David Carroll (vmlinuxz) Date: 2005-04-08 10:13 Message: Logged In: YES user_id=684143 I achieve the same results with the following code. ftp = FTP('ftp.cdrom.com') ftp.login('anonymous', 'anonymous@') ftp.retrlines('LIST') This code works just fine with your patched ftplib.py, or the old ftplib.py against all public internet ftp servers I have tried it on. It works on the server at work only sporadically,, and raising the protocol exception the rest of the time. I'm inclined to think that there may be a problem with the protocol implementation on the server side. The server software version is wu-2.6.2(1), and the only oddity I see with it is a pretty sizeable delay before giving the login. Up to 10 seconds. I'd love to get a pointer of where to take this debugging process next. Perhaps I need to use a sniffer on the connection or something. I would love to see what ftplib.py is getting back that is breaking it. ---------------------------------------------------------------------- Comment By: David Carroll (vmlinuxz) Date: 2005-04-07 09:51 Message: Logged In: YES user_id=684143 Thank you for leaving this open. I will try and get your example today. I'm too uneducated to realize what I'm doing wrong, so maybe you can use my ignorance for the common good =). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-05 04:31 Message: Logged In: YES user_id=80475 I fixed the IndexError. It should have been a protocol error. See Lib/ftplib.py 1.74. Leaving the bug open so the OP can make a more detailed post so we can see what the root issue is about. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 03:37 Message: Logged In: YES user_id=971153 It turns out that the code in plain text bug report is more or less readable, so I can see what you were trying to do... Seems like you are doing ftp.dir("LIST "+ftpDir) But ftplib docs: http://www.python.org/doc/current/lib/ftp-objects.html say that you need to specify the directory ftp.dir(ftpDir) so it seems that you are using ftp.dir() incorrectly.... But, even then IndexError seems like a wrong kind of exception for this kind of error... So, I guess, my original request for a working code snippet which would allow to reproduce the bug still stands.. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 02:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Fri Apr 8 13:54:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 13:54:49 2005 Subject: [ python-Bugs-1179168 ] IDLE bug - changing shortcuts Message-ID: Bugs item #1179168, was opened at 2005-04-08 13:54 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179168&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Przemysław Gocyła (bld_d16) Assigned to: Nobody/Anonymous (nobody) Summary: IDLE bug - changing shortcuts Initial Comment: On Python 2.4.1 on WinXP I have changed keyboard shortcut settings for indent/unindent. New shorcut was Tab/Shift+Tab. When I did it IDLE hanged up and I could not to launch it again. I have tried everything. Uninstall whole Python and then reinstall (tried both version 2.3 and 2.4). It just looks like from the change of shortcuts IDLE don't works. I tried to: import idlelib.PyShell idlelib.PyShell.main() and result with bug are in attachment Where is the information about IDLE shortcuts keeped ? Why IDLE remember this even after reinstall ? How to fix it up? Best regards, Przemek ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179168&group_id=5470 From noreply at sourceforge.net Fri Apr 8 18:17:54 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 18:18:16 2005 Subject: [ python-Bugs-1178145 ] urllib2.py assumes 206 is an error Message-ID: Bugs item #1178145, was opened at 2005-04-06 18:52 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178145&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Barry Alan Scott (barry-scott) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2.py assumes 206 is an error Initial Comment: I'm writting code that uses the Range header. The correct response is 206, but the urllib2.py is coded to treat any code that is not 200 as error. The correct code needs to treat 200 to 299 as success. The attached patch fixes the problem. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2005-04-08 12:17 Message: Logged In: YES user_id=764593 Please re-attach. SF didn't get the file. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178145&group_id=5470 From noreply at sourceforge.net Fri Apr 8 18:18:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 18:18:58 2005 Subject: [ python-Bugs-1178141 ] urllib.py overwrite HTTPError code with 200 Message-ID: Bugs item #1178141, was opened at 2005-04-06 18:48 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178141&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Barry Alan Scott (barry-scott) Assigned to: Nobody/Anonymous (nobody) Summary: urllib.py overwrite HTTPError code with 200 Initial Comment: I found this bug while trying to understand why a 404 Not Found error was reported as a 200 Not Found. Turns out the HTTPError's self.code is overwritten with 200 after the 404 was correctly assigned. The attached patch fixes the problem. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2005-04-08 12:18 Message: Logged In: YES user_id=764593 Please reattach (SF didn't catch the file) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178141&group_id=5470 From noreply at sourceforge.net Fri Apr 8 21:42:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 8 21:42:49 2005 Subject: [ python-Bugs-1179412 ] can't import thru cygwin symlink Message-ID: Bugs item #1179412, was opened at 2005-04-08 15:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179412&group_id=5470 Category: Windows Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: steveward (steveward) Assigned to: Nobody/Anonymous (nobody) Summary: can't import thru cygwin symlink Initial Comment: This may be a cygwin-specific problem: given foo.py: > ln -s foo.py bar.py > python Python 2.4 (#1, Dec 4 2004, 20:10:33) [GCC 3.3.3 (cygwin special)] on cygwin Type "help", "copyright", "credits" or "license" for more information. >>> import foo This is file foo.py. >>> import bar Traceback (most recent call last): File "", line 1, in ? ImportError: No module named bar Despite the problem with imports, most os.path utilities (exists, isfile, islink, isdir) work on cygwin symlinks. An exception is reailpath: realpath("bar.py") returns a path to the symlink, not to the real file. Suspecting this as a key to the import problem, I tried several recent python/cygwin release versions (all installed via cygwin's setup.exe). FIndings: Cygwin Python realpath Import 1.5.xx: 2.yy: Works? Works? --------- -------------- ----------- ---------- 1.5.14 2.4 NO NO 1.5.13 2.3.4 YES NO 1.5.14 2.3.4 YES NO 1.5.12 2.4 NO YES Neither bug shows up under Linux. The two problems seem uncorrelated, although it may be that each is due to some assumpion about symlink semantics that isn't true of the Cygwin implementation. Apologies if these problems have been previously submitted in a form my quick scan didn't identify. A corresponding note has been submitted to the cygwin mailing list. - Steve ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179412&group_id=5470 From noreply at sourceforge.net Sat Apr 9 04:34:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 9 04:34:05 2005 Subject: [ python-Bugs-1176012 ] compiler module didn't get updated for "class foo():pass" Message-ID: Bugs item #1176012, was opened at 2005-04-03 14:21 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 Category: Python Library Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: logistix (logistix) Assigned to: Brett Cannon (bcannon) Summary: compiler module didn't get updated for "class foo():pass" Initial Comment: Patch with fix and testcase is attached. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-08 19:34 Message: Logged In: YES user_id=357491 OK, committed in rev. XXX for Modules/parsermodule.c, rev. XXX for Lib/test/test_parser.py, rev. XXX for Lib/compiler/transformer.py, and rev. XXX for Lib/test/test_compiler.py . Only change from patch to what was committed was changing the formatting for parsermodule.c to meet the coding standard and a bug where an assignment was being made instead of a comparison. Closed as fixed. Thanks, logistix. ---------------------------------------------------------------------- Comment By: logistix (logistix) Date: 2005-04-03 15:01 Message: Logged In: YES user_id=699438 The parser module didn't get corrected either. Here is another patch with test case for that. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176012&group_id=5470 From noreply at sourceforge.net Sat Apr 9 10:09:39 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 9 10:09:46 2005 Subject: [ python-Bugs-1163244 ] Syntax error on large file with MBCS encoding Message-ID: Bugs item #1163244, was opened at 2005-03-14 22:20 Message generated for change (Comment added) made by tilinna You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163244&group_id=5470 Category: Parser/Compiler Group: Python 2.4 Status: Open Resolution: Accepted Priority: 7 Submitted By: Tim N. van der Leeuw (tnleeuw) Assigned to: Nobody/Anonymous (nobody) Summary: Syntax error on large file with MBCS encoding Initial Comment: Large files generated by make-py.py from the win32all extensions cannot be compiled by Python2.4.1rc1 - they give a syntax error. This is a regression from 2.3.5 (With Python2.4, the interpreter crashes. That is fixed now.) Removing the mbcs encoding line from the top of the file, compilation succeeds. File should be attached, as zip-file. Probably requires win32all extensions to be installed to be compiled / imported (generated using build 203 of the win32all extensions). ---------------------------------------------------------------------- Comment By: Timo Linna (tilinna) Date: 2005-04-09 11:09 Message: Logged In: YES user_id=1074183 Seems that the connection to n*512 blocks is very likely, and it's not just MBCS-related. I managed to reproduce this with a file that contains an ascii-coding declaration, close-to-1024 bytes section, extra crlf and a comment which raises a SyntaxError in Py2.4.1. Could this be linked to the new codec buffering code? See: www.python.org/sf/1178484 ---------------------------------------------------------------------- Comment By: Christos Georgiou (tzot) Date: 2005-03-21 15:34 Message: Logged In: YES user_id=539787 Could be irrelevant but... are the other block sizes close to n*512 (eg 1536 is 3*512) marks? ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2005-03-21 14:11 Message: Logged In: YES user_id=14198 I believe this is a different bug than the recent "long-lines" errors (see below). I can reproduce this with a file that uses neither long lines, nor any pywin32 extensions (2.4 branch, trunk) A Python source file containing: -- start snippet -- # -*- coding: mbcs -*- <1532 characters of code or comments> x = {} -- end snippet -- Will yield a SyntaxError when attempting to import the module. Running the module as a script does not provoke the error. To reproduce, there must be exactly 1532 characters where specified (see the attached file for a demo). Adding or removing even a single character will prevent the error. It is possible to replace characters with any others, including valid code, and still see the error - however, the number of characters must remain the same .cr/lf pairs can also be replaced with any other 2 characters. There are other "block sizes" that will provoke the error, but this is the only one I have nailed. Apart from the "block" of 1532 characters, the coding line and the blank line before the dict assignment also appear critical. Unlike the other characters in the block, this last cr/lf pair can not be replaced with comments. I can't provoke the error with other encodings (note there are no encoded characters in the sample - it is trivial). To reproduce, save the attached file on Windows and execute: > python -c "import foo2" Traceback (most recent call last): File "", line 1, in ? File "foo2.py", line 24 x = {} ^ SyntaxError: invalid syntax Note that Python 2.3 and earlier all work. Also note that "python foo2.py" also works. The code is clearly valid. Haven't tried to repro on Linux (mbcs isn't available there, and I can't get a test case that doesn't use it) Other pointers/notes: pywin32 bug 1085454 is related to long-lines, by all accounts that underlying error has been fixed - I can't verify this as pywin32 no longer generates insanely long lines. I can confirm Python bugs 1101726/1089395 still crashes Python 2.3+. I believe all (including this) are discrete bugs. [foo2.py is my attachment - ya gotta love sourceforge :)] ---------------------------------------------------------------------- Comment By: Christos Georgiou (tzot) Date: 2005-03-20 12:28 Message: Logged In: YES user_id=539787 Useful pointers: in Python-dev, this has been characterised as related to pywin32 bug 1085454. Also related to www.python.org/sf/1101726 and www.python.org/sf/1089395. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163244&group_id=5470 From noreply at sourceforge.net Sat Apr 9 12:33:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 9 12:33:48 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-09 12:33 Message: Logged In: YES user_id=21627 Why do you believe that _SGIAPI is defined? Please try compiling your program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-08 09:51 Message: Logged In: YES user_id=1254741 My test program that compiled correctly with "cc -c -g": #include int main () { void *p = setgroups; return 0; } The definition of setgroups in : #if _SGIAPI ... #if defined(_BSD_COMPAT) extern int setgroups(int, int *); #else /* !_BSD_COMPAT */ extern int setgroups(int, const gid_t *); #endif /* _BSD_COMPAT */ ... #endif /* _SGIAPI */ So it should catch one of these defines. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Sun Apr 10 00:10:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 00:10:50 2005 Subject: [ python-Bugs-1179957 ] Missing def'n of equality for set elements Message-ID: Bugs item #1179957, was opened at 2005-04-09 17:10 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179957&group_id=5470 Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: Missing def'n of equality for set elements Initial Comment: The documentation of the sets module doesn't describe the properties of set elements that they must have to properly work in sets (at least I didn't see it described). In creating a set of instances, I had to sort of stumble around and discover that the class needed to define both __eq__ and __hash__. Either alone didn't seem to work. Perhaps I've got it all wrong and there's a better way to do things. It would be nice if the docs described things officially though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179957&group_id=5470 From noreply at sourceforge.net Sun Apr 10 00:47:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 00:48:04 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 06:14 Message generated for change (Comment added) made by glchapman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-09 14:47 Message: Logged In: YES user_id=86307 Sorry to comment on a closed report, but perhaps this fix should not be limited only to cases where size is None. Today, I ran into a spurious syntax error when trying to import pythondoc (from http://effbot.org/downloads/pythondoc-2.1b3-20050325.zip). It turned out a \r was ending up in what looked to the parser like the middle of a line, presumably because a \n was dropped. Anyway, I applied the referenced patch to 2.4.1, except I left out the "size is None" condition, since I knew the tokenizer passes in a size param. With that change pythondoc import successfully. (Also, I just ran the test suite and nothing broke.) Since the size parameter is already documented as being passed to StreamReader.read (in codecs.py -- the HTML documentation needs to be updated), and since StreamReader.read says size is an approximate maximum, perhaps it's OK to read one extra byte. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-04 14:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Sun Apr 10 03:28:43 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 03:28:46 2005 Subject: [ python-Bugs-1180002 ] locale.format question Message-ID: Bugs item #1180002, was opened at 2005-04-10 01:28 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180002&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Andrew Ma (andrewma) Assigned to: Nobody/Anonymous (nobody) Summary: locale.format question Initial Comment: locale.format is returning send("234,5") rather than send ("2,345") as I was expecting. Is this a bug? The example is modified from the Tutorial 11.1 Output Formatting -------------------------------------------------------------------------- Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.setlocale(locale.LC_ALL, 'English_United States.1252') 'English_United States.1252' >>> data=2345 >>> locale.format('send("%d")', data, grouping = True) 'send("234,5")' >>> locale.format('%d', data, grouping = True) '2,345' >>> ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180002&group_id=5470 From noreply at sourceforge.net Sun Apr 10 07:51:45 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 07:51:47 2005 Subject: [ python-Bugs-1180002 ] locale.format question Message-ID: Bugs item #1180002, was opened at 2005-04-10 03:28 Message generated for change (Comment added) made by birkenfeld You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180002&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Andrew Ma (andrewma) Assigned to: Nobody/Anonymous (nobody) Summary: locale.format question Initial Comment: locale.format is returning send("234,5") rather than send ("2,345") as I was expecting. Is this a bug? The example is modified from the Tutorial 11.1 Output Formatting -------------------------------------------------------------------------- Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.setlocale(locale.LC_ALL, 'English_United States.1252') 'English_United States.1252' >>> data=2345 >>> locale.format('send("%d")', data, grouping = True) 'send("234,5")' >>> locale.format('%d', data, grouping = True) '2,345' >>> ---------------------------------------------------------------------- Comment By: Reinhold Birkenfeld (birkenfeld) Date: 2005-04-10 07:51 Message: Logged In: YES user_id=1188172 locale.format is implemented very "pragmatic". For example, you can't do locale.format('"1.%d."', 123) ("too many decimal points") though this should be supported. This is because it first formats the string with the normal "%" operation and then searches the whole string for decimal points to substitute. IMHO, format should first separate the % escapes from the format string, then format the numbers accordingly, and then use the % operation. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180002&group_id=5470 From noreply at sourceforge.net Sun Apr 10 13:10:37 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 13:10:39 2005 Subject: [ python-Bugs-1180147 ] test_posix fails on cygwin Message-ID: Bugs item #1180147, was opened at 2005-04-10 13:10 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180147&group_id=5470 Category: None Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Henrik Wist (wist) Assigned to: Nobody/Anonymous (nobody) Summary: test_posix fails on cygwin Initial Comment: $ python test/test_posix.py testNoArgFunctions (__main__.PosixTester) ... ERROR test_access (__main__.PosixTester) ... ok test_chdir (__main__.PosixTester) ... ok test_dup (__main__.PosixTester) ... ok test_dup2 (__main__.PosixTester) ... ok test_fdopen (__main__.PosixTester) ... ok test_fstat (__main__.PosixTester) ... ok test_fstatvfs (__main__.PosixTester) ... ok test_ftruncate (__main__.PosixTester) ... ok test_lsdir (__main__.PosixTester) ... ok test_pipe (__main__.PosixTester) ... ok test_stat (__main__.PosixTester) ... ok test_statvfs (__main__.PosixTester) ... ok test_strerror (__main__.PosixTester) ... ok test_tempnam (__main__.PosixTester) ... ok test_tmpfile (__main__.PosixTester) ... ok test_umask (__main__.PosixTester) ... ok test_utime (__main__.PosixTester) ... ok ====================================================================== ERROR: testNoArgFunctions (__main__.PosixTester) ---------------------------------------------------------------------- Traceback (most recent call last): File "test/test_posix.py", line 40, in testNoArgFunctions posix_func() OSError: [Errno 22] Invalid argument ---------------------------------------------------------------------- Ran 18 tests in 0.038s FAILED (errors=1) Traceback (most recent call last): File "test/test_posix.py", line 159, in ? test_main() File "test/test_posix.py", line 156, in test_main test_support.run_unittest(PosixTester) File "/usr/lib/python2.3/test/test_support.py", line 262, in run_unittest run_suite(suite, testclass) File "/usr/lib/python2.3/test/test_support.py", line 247, in run_suite raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "test/test_posix.py", line 40, in testNoArgFunctions posix_func() OSError: [Errno 22] Invalid argument This is with cygwin 1.5.12-1 and python 2.3.4-2. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180147&group_id=5470 From noreply at sourceforge.net Sun Apr 10 13:12:15 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 13:12:18 2005 Subject: [ python-Bugs-1180147 ] test_posix fails on cygwin Message-ID: Bugs item #1180147, was opened at 2005-04-10 13:10 Message generated for change (Comment added) made by wist You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180147&group_id=5470 Category: None Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Henrik Wist (wist) Assigned to: Nobody/Anonymous (nobody) Summary: test_posix fails on cygwin Initial Comment: $ python test/test_posix.py testNoArgFunctions (__main__.PosixTester) ... ERROR test_access (__main__.PosixTester) ... ok test_chdir (__main__.PosixTester) ... ok test_dup (__main__.PosixTester) ... ok test_dup2 (__main__.PosixTester) ... ok test_fdopen (__main__.PosixTester) ... ok test_fstat (__main__.PosixTester) ... ok test_fstatvfs (__main__.PosixTester) ... ok test_ftruncate (__main__.PosixTester) ... ok test_lsdir (__main__.PosixTester) ... ok test_pipe (__main__.PosixTester) ... ok test_stat (__main__.PosixTester) ... ok test_statvfs (__main__.PosixTester) ... ok test_strerror (__main__.PosixTester) ... ok test_tempnam (__main__.PosixTester) ... ok test_tmpfile (__main__.PosixTester) ... ok test_umask (__main__.PosixTester) ... ok test_utime (__main__.PosixTester) ... ok ====================================================================== ERROR: testNoArgFunctions (__main__.PosixTester) ---------------------------------------------------------------------- Traceback (most recent call last): File "test/test_posix.py", line 40, in testNoArgFunctions posix_func() OSError: [Errno 22] Invalid argument ---------------------------------------------------------------------- Ran 18 tests in 0.038s FAILED (errors=1) Traceback (most recent call last): File "test/test_posix.py", line 159, in ? test_main() File "test/test_posix.py", line 156, in test_main test_support.run_unittest(PosixTester) File "/usr/lib/python2.3/test/test_support.py", line 262, in run_unittest run_suite(suite, testclass) File "/usr/lib/python2.3/test/test_support.py", line 247, in run_suite raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "test/test_posix.py", line 40, in testNoArgFunctions posix_func() OSError: [Errno 22] Invalid argument This is with cygwin 1.5.12-1 and python 2.3.4-2. ---------------------------------------------------------------------- >Comment By: Henrik Wist (wist) Date: 2005-04-10 13:12 Message: Logged In: YES user_id=1256464 And, I forgot, this is with WinXP SP2 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180147&group_id=5470 From noreply at sourceforge.net Sun Apr 10 13:41:18 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 13:41:21 2005 Subject: [ python-Bugs-1180160 ] subprocess.Popen fails with closed stdout Message-ID: Bugs item #1180160, was opened at 2005-04-10 13:41 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180160&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: neuhauser (neuhauser) Assigned to: Nobody/Anonymous (nobody) Summary: subprocess.Popen fails with closed stdout Initial Comment: I have a piece of code that gets run in a script that has its stdout closed: import sys sys.stdout = sys.stderr c = subprocess.Popen (..., stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.STDOUT) and this is what I get: Sending SVNR/permissions Transmitting file data .svn: Commit failed (details follow): svn: 'pre-commit' hook failed with error output: Traceback (most recent call last): (...) File ".../__init__.py", line 40, in run stderr = subprocess.STDOUT) File "/usr/local/lib/python2.4/subprocess.py", line 554, in __init__ errread, errwrite) File "/usr/local/lib/python2.4/subprocess.py", line 986, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This is the child_traceback: File "/usr/local/lib/python2.4/subprocess.py", line 955, in _execute_child File "/usr/local/lib/python2.4/os.py", line 341, in execvp File "/usr/local/lib/python2.4/os.py", line 379, in _execvpe func(fullname, *argrest) OSError: [Errno 2] No such file or directory http://docs.python.org/lib/node230.html claims that "PIPE indicates that a new pipe to the child should be created" which doesn't seem to be the case. Subversion code that runs the script can be seen at http://svn.collab.net/viewcvs/svn/trunk/subversion/libsvn_repos/hooks.c (run_hook_cmd()). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180160&group_id=5470 From noreply at sourceforge.net Sun Apr 10 15:10:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 15:10:54 2005 Subject: [ python-Bugs-1180193 ] broken pyc files Message-ID: Bugs item #1180193, was opened at 2005-04-10 13:10 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Armin Rigo (arigo) Assigned to: Nobody/Anonymous (nobody) Summary: broken pyc files Initial Comment: In a number of situations, the .pyc files can become "corrupted" in a subtle way: the co_filename attribute of the code objects it contains become wrong. This can occur if we move or rename directories, or if we access the same set of files from two different locations (e.g. over NFS). This corruption doesn't prevent the .pyc files from working, but the interpreter looses the reference to the source file. It causes trouble in tracebacks, in the inspect module, etc. A simple fix would be to use the following logic when importing a .py file: if there is a corresponding .pyc file, in addition to checking the timestamp, check the co_filename attribute of the loaded object. If it doesn't point to the original .py file, discard the code object and ignore the .pyc file. Alternatively, we could force all co_filenames to point to the .py file when loading the .pyc file. I'll write a patch for whichever alternative seems better. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180193&group_id=5470 From noreply at sourceforge.net Sun Apr 10 16:55:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 16:55:38 2005 Subject: [ python-Bugs-1180002 ] locale.format question Message-ID: Bugs item #1180002, was opened at 2005-04-10 01:28 Message generated for change (Comment added) made by andrewma You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180002&group_id=5470 Category: Python Library Group: Python 2.4 >Status: Closed >Resolution: Remind Priority: 5 Submitted By: Andrew Ma (andrewma) Assigned to: Nobody/Anonymous (nobody) Summary: locale.format question Initial Comment: locale.format is returning send("234,5") rather than send ("2,345") as I was expecting. Is this a bug? The example is modified from the Tutorial 11.1 Output Formatting -------------------------------------------------------------------------- Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import locale >>> locale.setlocale(locale.LC_ALL, 'English_United States.1252') 'English_United States.1252' >>> data=2345 >>> locale.format('send("%d")', data, grouping = True) 'send("234,5")' >>> locale.format('%d', data, grouping = True) '2,345' >>> ---------------------------------------------------------------------- >Comment By: Andrew Ma (andrewma) Date: 2005-04-10 14:55 Message: Logged In: YES user_id=621709 Berk, I followed your advice using 'send("%s")' % locale.format("%d", data, grouping=True) and solved my problem. Too bad there is no way to add your advice to the Python documentation. I am putting the status to remind, in case anybody (including me) wants to fix this later. Thanks. ---------------------------------------------------------------------- Comment By: Reinhold Birkenfeld (birkenfeld) Date: 2005-04-10 05:51 Message: Logged In: YES user_id=1188172 locale.format is implemented very "pragmatic". For example, you can't do locale.format('"1.%d."', 123) ("too many decimal points") though this should be supported. This is because it first formats the string with the normal "%" operation and then searches the whole string for decimal points to substitute. IMHO, format should first separate the % escapes from the format string, then format the numbers accordingly, and then use the % operation. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180002&group_id=5470 From noreply at sourceforge.net Sun Apr 10 17:20:05 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 17:20:08 2005 Subject: [ python-Bugs-1180237 ] Python keeps file references after calling close methode Message-ID: Bugs item #1180237, was opened at 2005-04-10 17:20 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180237&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Eelco (eternia) Assigned to: Nobody/Anonymous (nobody) Summary: Python keeps file references after calling close methode Initial Comment: I found this bug using a python script that: - first mounts a partition (os.system("mount") etc) - change a few files on this partition (f = open (); f.write; f.close) - umounts the partition (os.system("umount") etc) Strangely, the umount didn't work because of a filesystem busy error. Using fuser and lsof i traced this being busy back to the script itself. This is strange behavior because after changing the files on the mounted partition the close method was called which should close all references to the file on the partition. Finally the solution was to do f = 0. So if python has closed a file on a mount a open reference to that file will keep to exist until the script has ended or until the file object is nullified. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180237&group_id=5470 From noreply at sourceforge.net Sun Apr 10 18:44:28 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 18:44:31 2005 Subject: [ python-Bugs-1180267 ] expanding platform module and making it work as it should Message-ID: Bugs item #1180267, was opened at 2005-04-10 19:44 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nikos Kouremenos (nkour) Assigned to: Nobody/Anonymous (nobody) Summary: expanding platform module and making it work as it should Initial Comment: platform.release() (which is supposed to return the Name of Windows) also does not work as it should in some versions of windows I tried (xp pro sp1). Luckily to print more than Windows (eg. print Windows XP or Windows 2000 etc) you can have a look at this http://www.brunningonline.net/simon/blog/archives/001168.html of Simon Brunning also only debian, mdk and redhat is scanned for GNU/Linux ?? why even bother then? I think that PSL is good but this module is has hell of limitations. At least dont' make anyone write this: import os distro_info = { 'Arch Linux': '/etc/arch-release', 'Aurox Linux': '/etc/aurox-release', 'Conectiva Linux': '/etc/conectiva-release', 'Debian GNU/Linux': '/etc/debian_release', 'Debian GNU/Linux': '/etc/debian_version', 'Fedora Linux': '/etc/fedora-release', 'Gentoo Linux': '/etc/gentoo-release', 'Mandrake Linux': '/etc/mandrake-release', 'Slackware Linux': '/etc/slackware-release', 'Slackware Linux': '/etc/slackware-version', 'Solaris/Sparc': '/etc/release', 'Sun JDS': '/etc/sun-release', 'Novell SUSE Linux': '/etc/SuSE-release', 'PLD Linux': '/etc/pld-release', 'SUSE Linux': '/etc/SuSE-release', 'Yellow Dog Linux': '/etc/yellowdog-release', # many distros use the /etc/redhat-release for compatibility # so Redhat is the last 'Redhat Linux': '/etc/redhat-release'} def get_os_info(): if os.name =='nt': win_version = { (1, 4, 0): "95", (1, 4, 10): "98", (1, 4, 90): "ME", (2, 4, 0): "NT", (2, 5, 0): "2000", (2, 5, 1): "XP" }[os.sys.getwindowsversion()[3], os.sys.getwindowsversion()[0], os.sys.getwindowsversion()[1]] return 'Windows' + ' ' + win_version elif os.name =='posix': executable = 'lsb_release' params = ' --id --codename --release --short' for path in os.environ['PATH'].split(':'): full_path_to_executable = os.path.join(path, executable) if os.path.exists(full_path_to_executable): command = executable + params child_stdin, child_stdout = os.popen2(command) output = child_stdout.readline().strip() child_stdout.close() child_stdin.close() return output # lsb_release executable not available, so parse files for distro in distro_info: path_to_file = distro_info[distro] if os.path.exists(path_to_file): file = open(path_to_file) text = file.read().strip() file.close() if path_to_file.endswith('version'): text = distro + ' ' + text return text print get_os_info() Thank you ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 From noreply at sourceforge.net Sun Apr 10 18:46:04 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 18:46:08 2005 Subject: [ python-Bugs-1180267 ] expanding platform module and making it work as it should Message-ID: Bugs item #1180267, was opened at 2005-04-10 19:44 Message generated for change (Comment added) made by nkour You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nikos Kouremenos (nkour) Assigned to: Nobody/Anonymous (nobody) Summary: expanding platform module and making it work as it should Initial Comment: platform.release() (which is supposed to return the Name of Windows) also does not work as it should in some versions of windows I tried (xp pro sp1). Luckily to print more than Windows (eg. print Windows XP or Windows 2000 etc) you can have a look at this http://www.brunningonline.net/simon/blog/archives/001168.html of Simon Brunning also only debian, mdk and redhat is scanned for GNU/Linux ?? why even bother then? I think that PSL is good but this module is has hell of limitations. At least dont' make anyone write this: import os distro_info = { 'Arch Linux': '/etc/arch-release', 'Aurox Linux': '/etc/aurox-release', 'Conectiva Linux': '/etc/conectiva-release', 'Debian GNU/Linux': '/etc/debian_release', 'Debian GNU/Linux': '/etc/debian_version', 'Fedora Linux': '/etc/fedora-release', 'Gentoo Linux': '/etc/gentoo-release', 'Mandrake Linux': '/etc/mandrake-release', 'Slackware Linux': '/etc/slackware-release', 'Slackware Linux': '/etc/slackware-version', 'Solaris/Sparc': '/etc/release', 'Sun JDS': '/etc/sun-release', 'Novell SUSE Linux': '/etc/SuSE-release', 'PLD Linux': '/etc/pld-release', 'SUSE Linux': '/etc/SuSE-release', 'Yellow Dog Linux': '/etc/yellowdog-release', # many distros use the /etc/redhat-release for compatibility # so Redhat is the last 'Redhat Linux': '/etc/redhat-release'} def get_os_info(): if os.name =='nt': win_version = { (1, 4, 0): "95", (1, 4, 10): "98", (1, 4, 90): "ME", (2, 4, 0): "NT", (2, 5, 0): "2000", (2, 5, 1): "XP" }[os.sys.getwindowsversion()[3], os.sys.getwindowsversion()[0], os.sys.getwindowsversion()[1]] return 'Windows' + ' ' + win_version elif os.name =='posix': executable = 'lsb_release' params = ' --id --codename --release --short' for path in os.environ['PATH'].split(':'): full_path_to_executable = os.path.join(path, executable) if os.path.exists(full_path_to_executable): command = executable + params child_stdin, child_stdout = os.popen2(command) output = child_stdout.readline().strip() child_stdout.close() child_stdin.close() return output # lsb_release executable not available, so parse files for distro in distro_info: path_to_file = distro_info[distro] if os.path.exists(path_to_file): file = open(path_to_file) text = file.read().strip() file.close() if path_to_file.endswith('version'): text = distro + ' ' + text return text print get_os_info() Thank you ---------------------------------------------------------------------- >Comment By: Nikos Kouremenos (nkour) Date: 2005-04-10 19:46 Message: Logged In: YES user_id=865368 identation went to take a walk :( I'm sorry look here: http://nkour.blogspot.com/2005/03/python-script-to-detect-gnulinux.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 From noreply at sourceforge.net Sun Apr 10 19:20:37 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 19:20:39 2005 Subject: [ python-Bugs-1179957 ] Missing def'n of equality for set elements Message-ID: Bugs item #1179957, was opened at 2005-04-09 17:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179957&group_id=5470 Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: Missing def'n of equality for set elements Initial Comment: The documentation of the sets module doesn't describe the properties of set elements that they must have to properly work in sets (at least I didn't see it described). In creating a set of instances, I had to sort of stumble around and discover that the class needed to define both __eq__ and __hash__. Either alone didn't seem to work. Perhaps I've got it all wrong and there's a better way to do things. It would be nice if the docs described things officially though. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-10 12:20 Message: Logged In: YES user_id=80475 It does say that sets are implemented using dictionaries. That implies that the requirements for set elements are the same as for dictionary keys. Will see if I can add some clarifying wording. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179957&group_id=5470 From noreply at sourceforge.net Sun Apr 10 19:30:04 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 10 19:30:07 2005 Subject: [ python-Bugs-1179957 ] Missing def'n of equality for set elements Message-ID: Bugs item #1179957, was opened at 2005-04-09 17:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179957&group_id=5470 Category: Documentation Group: Python 2.5 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: Missing def'n of equality for set elements Initial Comment: The documentation of the sets module doesn't describe the properties of set elements that they must have to properly work in sets (at least I didn't see it described). In creating a set of instances, I had to sort of stumble around and discover that the class needed to define both __eq__ and __hash__. Either alone didn't seem to work. Perhaps I've got it all wrong and there's a better way to do things. It would be nice if the docs described things officially though. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-10 12:30 Message: Logged In: YES user_id=80475 Updated both libstdtypes.tex and libsets.tex with explicit documentation for element requirements. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-10 12:20 Message: Logged In: YES user_id=80475 It does say that sets are implemented using dictionaries. That implies that the requirements for set elements are the same as for dictionary keys. Will see if I can add some clarifying wording. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179957&group_id=5470 From noreply at sourceforge.net Mon Apr 11 00:55:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 00:55:16 2005 Subject: [ python-Bugs-1180392 ] StringIO's docs should mention overwriting of initial value Message-ID: Bugs item #1180392, was opened at 2005-04-10 17:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180392&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Leif K-Brooks (eurleif) Assigned to: Nobody/Anonymous (nobody) Summary: StringIO's docs should mention overwriting of initial value Initial Comment: StringIO's initial file position is always set to 0, rather than being set to the length of the initial text. This isn't mentioned in the documentation, which could cause confusion. I've attached some text the docs could use. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180392&group_id=5470 From noreply at sourceforge.net Mon Apr 11 00:56:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 00:56:59 2005 Subject: [ python-Bugs-1180392 ] StringIO's docs should mention overwriting of initial value/ Message-ID: Bugs item #1180392, was opened at 2005-04-10 17:55 Message generated for change (Settings changed) made by eurleif You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180392&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Leif K-Brooks (eurleif) Assigned to: Nobody/Anonymous (nobody) >Summary: StringIO's docs should mention overwriting of initial value/ Initial Comment: StringIO's initial file position is always set to 0, rather than being set to the length of the initial text. This isn't mentioned in the documentation, which could cause confusion. I've attached some text the docs could use. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180392&group_id=5470 From noreply at sourceforge.net Mon Apr 11 03:04:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 03:04:58 2005 Subject: [ python-Bugs-1180392 ] StringIO's docs should mention overwriting of initial value/ Message-ID: Bugs item #1180392, was opened at 2005-04-10 17:55 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180392&group_id=5470 Category: Documentation Group: Feature Request >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Leif K-Brooks (eurleif) Assigned to: Nobody/Anonymous (nobody) Summary: StringIO's docs should mention overwriting of initial value/ Initial Comment: StringIO's initial file position is always set to 0, rather than being set to the length of the initial text. This isn't mentioned in the documentation, which could cause confusion. I've attached some text the docs could use. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-10 20:04 Message: Logged In: YES user_id=80475 Okay, I added an explicit comment that the initial file position is zero even when the object has been initialized with a string. FWIW, I recommend not using initialization when also using a write method. That will not allow a seemless conversion to cStringIO where intialized objects are read-only. A more scalable technique is to create an empty object and then call write() to initialize it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180392&group_id=5470 From noreply at sourceforge.net Mon Apr 11 06:26:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 06:26:42 2005 Subject: [ python-Bugs-1180470 ] BaseHTTPServer uses deprecated mimetools.Message Message-ID: Bugs item #1180470, was opened at 2005-04-11 04:26 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180470&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Paul Jimenez (paulj) Assigned to: Nobody/Anonymous (nobody) Summary: BaseHTTPServer uses deprecated mimetools.Message Initial Comment: BaseHTTPServer used a deprecated (as of 2.3) class: mimetools.Message. cgi.py also uses it, but that's getting fixed. If only there was just a single API for writing webapps. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180470&group_id=5470 From noreply at sourceforge.net Mon Apr 11 11:13:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 11:13:45 2005 Subject: [ python-Bugs-1168983 ] ftplib.py string index out of range Message-ID: Bugs item #1168983, was opened at 2005-03-23 10:05 Message generated for change (Comment added) made by vmlinuxz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: Accepted Priority: 5 Submitted By: David Carroll (vmlinuxz) Assigned to: Raymond Hettinger (rhettinger) Summary: ftplib.py string index out of range Initial Comment: The following code works great on all of the 2.3.x releases I have used. def ftpFetch(localPath,remoteFileDate,ftpSite,ftpPass,ftpDir,ftpUser): print "Fetching split_mgr report" fileList=[] processList=[] ftp = FTP(ftpSite) ftp.login(ftpUser,ftpPass) ftp.dir('LIST '+ftpDir,fileList.append) for x in range(len(fileList)): if (string.find(fileList[x],remoteFileDate) != -1): print fileList[x] TMP=fileList[x] output=localPath+str(TMP[45:]) print output processList.append(output) print processList outfile = open(output,'w') ftp.retrbinary('RETR '+ftpDir+TMP[45:], open(output, 'wb').write) ftp.quit() print processList return processList However I get the following error under 2.4 Traceback (most recent call last): File "C:\Python24\lib\site-packages\PythonCard\widget.py", line 417, in _dispatch handler(background, aWxEvent) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\readwaitreport.py", line 61, in on_btnRun_command mainRwclassLoop(self, mm+dd+yy, yesterday) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 39, in mainRwclassLoop processList = ftpFetch(localPath,"split_mgr."+date[0:4],ftpSite,ftpPass,ftpDir,ftpUser) File "C:\Documents and Settings\PROTECTED\Desktop\ReadWaitReport\ReadWaitReport\rwclass.py", line 173, in ftpFetch ftp.dir('LIST '+ftpDir,fileList.append) File "C:\Python24\lib\ftplib.py", line 464, in dir self.retrlines(cmd, func) File "C:\Python24\lib\ftplib.py", line 396, in retrlines conn = self.transfercmd(cmd) File "C:\Python24\lib\ftplib.py", line 345, in transfercmd return self.ntransfercmd(cmd, rest)[0] File "C:\Python24\lib\ftplib.py", line 328, in ntransfercmd if resp[0] != '1': IndexError: string index out of range https://lists.dulug.duke.edu/pipermail/yum/2004-August/005067.html discusses a similar problem under Linux in the YUM script, I have reproduced this error under 2000, and XP. I'm a fairly new programmer, and thus very new to python so I hope this is enough information. I will try and keep track of this and help out with more information in any capacity I can. Thank you for all your hard work and dedication. David Carroll ---------------------------------------------------------------------- >Comment By: David Carroll (vmlinuxz) Date: 2005-04-11 09:13 Message: Logged In: YES user_id=684143 I'm not sure if this needs any more clarification, but for accuracy I'm attaching a file with the working and non working code segments, and the level 2 debug. This code was run on 2.3, using your new ftplib modifications, and the distributed ftplib. The FTP server is wu-2.6.2(1) running on a SCO system (bleh). The code was tested in PASV and PORT mode. I'm new to this so I don't know if I should close this or the patch submitter. If you need anything else from me let me know. David ---------------------------------------------------------------------- Comment By: David Carroll (vmlinuxz) Date: 2005-04-08 10:13 Message: Logged In: YES user_id=684143 I achieve the same results with the following code. ftp = FTP('ftp.cdrom.com') ftp.login('anonymous', 'anonymous@') ftp.retrlines('LIST') This code works just fine with your patched ftplib.py, or the old ftplib.py against all public internet ftp servers I have tried it on. It works on the server at work only sporadically,, and raising the protocol exception the rest of the time. I'm inclined to think that there may be a problem with the protocol implementation on the server side. The server software version is wu-2.6.2(1), and the only oddity I see with it is a pretty sizeable delay before giving the login. Up to 10 seconds. I'd love to get a pointer of where to take this debugging process next. Perhaps I need to use a sniffer on the connection or something. I would love to see what ftplib.py is getting back that is breaking it. ---------------------------------------------------------------------- Comment By: David Carroll (vmlinuxz) Date: 2005-04-07 09:51 Message: Logged In: YES user_id=684143 Thank you for leaving this open. I will try and get your example today. I'm too uneducated to realize what I'm doing wrong, so maybe you can use my ignorance for the common good =). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-05 04:31 Message: Logged In: YES user_id=80475 I fixed the IndexError. It should have been a protocol error. See Lib/ftplib.py 1.74. Leaving the bug open so the OP can make a more detailed post so we can see what the root issue is about. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 03:37 Message: Logged In: YES user_id=971153 It turns out that the code in plain text bug report is more or less readable, so I can see what you were trying to do... Seems like you are doing ftp.dir("LIST "+ftpDir) But ftplib docs: http://www.python.org/doc/current/lib/ftp-objects.html say that you need to specify the directory ftp.dir(ftpDir) so it seems that you are using ftp.dir() incorrectly.... But, even then IndexError seems like a wrong kind of exception for this kind of error... So, I guess, my original request for a working code snippet which would allow to reproduce the bug still stands.. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-05 02:12 Message: Logged In: YES user_id=971153 Could you submit a minimalistic piece of code which illustrates the problem? As well as any input data which are required to reproduce it? (SourceForge garbled your original example) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1168983&group_id=5470 From noreply at sourceforge.net Mon Apr 11 14:29:06 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 14:29:08 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by zosh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: zosh (zosh) Date: 2005-04-11 14:29 Message: Logged In: YES user_id=1254741 I believe _SIGAPI is defined beacuse cc should report a non-existent "setgroups" otherwise. Compiling my test program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 resulted in a failure: cc-1020 cc: ERROR File = t.c, Line = 5 The identifier "setgroups" is undefined. void *p = setgroups; ^ ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-09 12:33 Message: Logged In: YES user_id=21627 Why do you believe that _SGIAPI is defined? Please try compiling your program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-08 09:51 Message: Logged In: YES user_id=1254741 My test program that compiled correctly with "cc -c -g": #include int main () { void *p = setgroups; return 0; } The definition of setgroups in : #if _SGIAPI ... #if defined(_BSD_COMPAT) extern int setgroups(int, int *); #else /* !_BSD_COMPAT */ extern int setgroups(int, const gid_t *); #endif /* _BSD_COMPAT */ ... #endif /* _SGIAPI */ So it should catch one of these defines. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Mon Apr 11 14:38:39 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 14:38:42 2005 Subject: [ python-Bugs-1180267 ] expanding platform module and making it work as it should Message-ID: Bugs item #1180267, was opened at 2005-04-10 19:44 Message generated for change (Comment added) made by nkour You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nikos Kouremenos (nkour) Assigned to: Nobody/Anonymous (nobody) Summary: expanding platform module and making it work as it should Initial Comment: platform.release() (which is supposed to return the Name of Windows) also does not work as it should in some versions of windows I tried (xp pro sp1). Luckily to print more than Windows (eg. print Windows XP or Windows 2000 etc) you can have a look at this http://www.brunningonline.net/simon/blog/archives/001168.html of Simon Brunning also only debian, mdk and redhat is scanned for GNU/Linux ?? why even bother then? I think that PSL is good but this module is has hell of limitations. At least dont' make anyone write this: import os distro_info = { 'Arch Linux': '/etc/arch-release', 'Aurox Linux': '/etc/aurox-release', 'Conectiva Linux': '/etc/conectiva-release', 'Debian GNU/Linux': '/etc/debian_release', 'Debian GNU/Linux': '/etc/debian_version', 'Fedora Linux': '/etc/fedora-release', 'Gentoo Linux': '/etc/gentoo-release', 'Mandrake Linux': '/etc/mandrake-release', 'Slackware Linux': '/etc/slackware-release', 'Slackware Linux': '/etc/slackware-version', 'Solaris/Sparc': '/etc/release', 'Sun JDS': '/etc/sun-release', 'Novell SUSE Linux': '/etc/SuSE-release', 'PLD Linux': '/etc/pld-release', 'SUSE Linux': '/etc/SuSE-release', 'Yellow Dog Linux': '/etc/yellowdog-release', # many distros use the /etc/redhat-release for compatibility # so Redhat is the last 'Redhat Linux': '/etc/redhat-release'} def get_os_info(): if os.name =='nt': win_version = { (1, 4, 0): "95", (1, 4, 10): "98", (1, 4, 90): "ME", (2, 4, 0): "NT", (2, 5, 0): "2000", (2, 5, 1): "XP" }[os.sys.getwindowsversion()[3], os.sys.getwindowsversion()[0], os.sys.getwindowsversion()[1]] return 'Windows' + ' ' + win_version elif os.name =='posix': executable = 'lsb_release' params = ' --id --codename --release --short' for path in os.environ['PATH'].split(':'): full_path_to_executable = os.path.join(path, executable) if os.path.exists(full_path_to_executable): command = executable + params child_stdin, child_stdout = os.popen2(command) output = child_stdout.readline().strip() child_stdout.close() child_stdin.close() return output # lsb_release executable not available, so parse files for distro in distro_info: path_to_file = distro_info[distro] if os.path.exists(path_to_file): file = open(path_to_file) text = file.read().strip() file.close() if path_to_file.endswith('version'): text = distro + ' ' + text return text print get_os_info() Thank you ---------------------------------------------------------------------- >Comment By: Nikos Kouremenos (nkour) Date: 2005-04-11 15:38 Message: Logged In: YES user_id=865368 ok nevermind the platform.release() part for windows name. it seems to work now.. :$ ---------------------------------------------------------------------- Comment By: Nikos Kouremenos (nkour) Date: 2005-04-10 19:46 Message: Logged In: YES user_id=865368 identation went to take a walk :( I'm sorry look here: http://nkour.blogspot.com/2005/03/python-script-to-detect-gnulinux.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 From noreply at sourceforge.net Mon Apr 11 18:00:28 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 18:00:32 2005 Subject: [ python-Bugs-1078245 ] Python2.4: building '_socket' extension fails with `INET_ADD Message-ID: Bugs item #1078245, was opened at 2004-12-03 05:16 Message generated for change (Comment added) made by ggarrett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Str?der (stroeder) Assigned to: Nobody/Anonymous (nobody) Summary: Python2.4: building '_socket' extension fails with `INET_ADD Initial Comment: HI! I'm trying to build Python2.4 on a rather old Debian machine. Building _socket fails (see below) although I tried to use configure --disable-ipv6 I've added #define INET_ADDRSTRLEN 16 to Modules/socketmodule.h which made it work. There were no problems compiling Python2.2 on the very same machine. Ciao, Michael. ------------------------------ snip ------------------------------ building '_socket' extension gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/tmp/Python-2.4/./Include -I/user/W3311/local/include -I/usr/local/include -I/tmp/Python-2.4/Include -I/tmp/Python-2.4 -c /tmp/Python-2.4/Modules/socketmodule.c -o build/t emp.linux-i686-2.4/socketmodule.o /tmp/Python-2.4/Modules/socketmodule.c: In function `socket_inet_ntop': /tmp/Python-2.4/Modules/socketmodule.c:3350: `INET_ADDRSTRLEN' undeclared (first use this function) /tmp/Python-2.4/Modules/socketmodule.c:3350: (Each undeclared identifier is reported only once /tmp/Python-2.4/Modules/socketmodule.c:3350: for each function it appears in.) /tmp/Python-2.4/Modules/socketmodule.c:3350: size of array `ip' has non-integer type ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-11 11:00 Message: Logged In: YES user_id=377412 I have the same problem on Solaris 7. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-01-06 14:35 Message: Logged In: YES user_id=21627 There are already a few attempts to define ADDRSTRLEN if it is missing. Maybe we should just merge them into one, and define it if it is not defined, independent of the system. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-13 07:19 Message: Logged In: YES user_id=29957 How old a debian release is this? It sounds like it's a just a broken system header, and I'm not entirely convinced that adding this to the already-large collection of #ifdefs in socketmodule is a path to happiness. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 From noreply at sourceforge.net Mon Apr 11 18:37:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 18:37:55 2005 Subject: [ python-Bugs-1078245 ] Python2.4: building '_socket' extension fails with `INET_ADD Message-ID: Bugs item #1078245, was opened at 2004-12-03 05:16 Message generated for change (Comment added) made by ggarrett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Str?der (stroeder) Assigned to: Nobody/Anonymous (nobody) Summary: Python2.4: building '_socket' extension fails with `INET_ADD Initial Comment: HI! I'm trying to build Python2.4 on a rather old Debian machine. Building _socket fails (see below) although I tried to use configure --disable-ipv6 I've added #define INET_ADDRSTRLEN 16 to Modules/socketmodule.h which made it work. There were no problems compiling Python2.2 on the very same machine. Ciao, Michael. ------------------------------ snip ------------------------------ building '_socket' extension gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/tmp/Python-2.4/./Include -I/user/W3311/local/include -I/usr/local/include -I/tmp/Python-2.4/Include -I/tmp/Python-2.4 -c /tmp/Python-2.4/Modules/socketmodule.c -o build/t emp.linux-i686-2.4/socketmodule.o /tmp/Python-2.4/Modules/socketmodule.c: In function `socket_inet_ntop': /tmp/Python-2.4/Modules/socketmodule.c:3350: `INET_ADDRSTRLEN' undeclared (first use this function) /tmp/Python-2.4/Modules/socketmodule.c:3350: (Each undeclared identifier is reported only once /tmp/Python-2.4/Modules/socketmodule.c:3350: for each function it appears in.) /tmp/Python-2.4/Modules/socketmodule.c:3350: size of array `ip' has non-integer type ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-11 11:37 Message: Logged In: YES user_id=377412 I got: building '_socket' extension gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -fno-strict-aliasing -I. -I/opt/src/Python-2.4.1/./Include -I/usr/local/include -I/opt/src/Python-2.4.1/Include -I/opt/src/Python-2.4.1 -c /opt/src/Python-2.4.1/Modules/socketmodule.c -o build/temp.solaris-2.7-sun4u-2.4/socketmodule.o In file included from /opt/src/Python-2.4.1/Modules/socketmodule.c:315: /opt/src/Python-2.4.1/Modules/getnameinfo.c: In function `fake_getnameinfo': /opt/src/Python-2.4.1/Modules/getnameinfo.c:173: warning: implicit declaration of function `inet_ntop' /opt/src/Python-2.4.1/Modules/getnameinfo.c:174: warning: comparison between pointer and integer /opt/src/Python-2.4.1/Modules/getnameinfo.c:206: warning: comparison between pointer and integer /opt/src/Python-2.4.1/Modules/socketmodule.c: In function `socket_inet_ntop': /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: `INET_ADDRSTRLEN' undeclared (first use in this function) /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: (Each undeclared identifier is reported only once /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: for each function it appears in.) /opt/src/Python-2.4.1/Modules/socketmodule.c:3380: warning: assignment makes pointer from integer without a cast /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: warning: unused variable `ip' My users then see... # regtester.py ../ini/tester.ini main Traceback (most recent call last): File "./regtester.py", line 24, in ? import core_engine, core_event, core_action, logger File "/etsi/tst/regtester.1.0.0/scripts/core_engine.py", line 19, in ? import time, Queue, sched, copy, string, threading, asyncore, socket File "/usr/local/lib/python2.4/asyncore.py", line 51, in ? import socket File "/usr/local/lib/python2.4/socket.py", line 45, in ? import _socket ImportError: No module named _socket I try your work-around. Doesn't help. ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-11 11:00 Message: Logged In: YES user_id=377412 I have the same problem on Solaris 7. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-01-06 14:35 Message: Logged In: YES user_id=21627 There are already a few attempts to define ADDRSTRLEN if it is missing. Maybe we should just merge them into one, and define it if it is not defined, independent of the system. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-13 07:19 Message: Logged In: YES user_id=29957 How old a debian release is this? It sounds like it's a just a broken system header, and I'm not entirely convinced that adding this to the already-large collection of #ifdefs in socketmodule is a path to happiness. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 From noreply at sourceforge.net Mon Apr 11 21:53:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 21:53:25 2005 Subject: [ python-Bugs-1180997 ] lax error-checking in new-in-2.4 marshal stuff Message-ID: Bugs item #1180997, was opened at 2005-04-11 20:53 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180997&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hudson (mwh) Assigned to: Martin v. L?wis (loewis) Summary: lax error-checking in new-in-2.4 marshal stuff Initial Comment: I realise one of the points of the TYPE_STRINGREF and so on stuff was efficiency, but: >>> marshal.loads('R') # TYPE_STRINGREF == 'R' Segmentation fault ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180997&group_id=5470 From noreply at sourceforge.net Mon Apr 11 23:21:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 23:21:52 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 23:21 Message: Logged In: YES user_id=21627 Hmm. You say "because cc should report a non-existent setgroups otherwise"; then you bring an error message '"setgroups" is undefined". Doesn't this imply that _SGIAPI (not _SIGAPI) is indeed not defined, in the case of Python, as well as the case of the various -D options? ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-11 14:29 Message: Logged In: YES user_id=1254741 I believe _SIGAPI is defined beacuse cc should report a non-existent "setgroups" otherwise. Compiling my test program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 resulted in a failure: cc-1020 cc: ERROR File = t.c, Line = 5 The identifier "setgroups" is undefined. void *p = setgroups; ^ ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-09 12:33 Message: Logged In: YES user_id=21627 Why do you believe that _SGIAPI is defined? Please try compiling your program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-08 09:51 Message: Logged In: YES user_id=1254741 My test program that compiled correctly with "cc -c -g": #include int main () { void *p = setgroups; return 0; } The definition of setgroups in : #if _SGIAPI ... #if defined(_BSD_COMPAT) extern int setgroups(int, int *); #else /* !_BSD_COMPAT */ extern int setgroups(int, const gid_t *); #endif /* _BSD_COMPAT */ ... #endif /* _SGIAPI */ So it should catch one of these defines. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Mon Apr 11 23:23:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 23:23:43 2005 Subject: [ python-Bugs-1078245 ] Python2.4: building '_socket' extension fails with `INET_ADD Message-ID: Bugs item #1078245, was opened at 2004-12-03 12:16 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Str?der (stroeder) Assigned to: Nobody/Anonymous (nobody) Summary: Python2.4: building '_socket' extension fails with `INET_ADD Initial Comment: HI! I'm trying to build Python2.4 on a rather old Debian machine. Building _socket fails (see below) although I tried to use configure --disable-ipv6 I've added #define INET_ADDRSTRLEN 16 to Modules/socketmodule.h which made it work. There were no problems compiling Python2.2 on the very same machine. Ciao, Michael. ------------------------------ snip ------------------------------ building '_socket' extension gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/tmp/Python-2.4/./Include -I/user/W3311/local/include -I/usr/local/include -I/tmp/Python-2.4/Include -I/tmp/Python-2.4 -c /tmp/Python-2.4/Modules/socketmodule.c -o build/t emp.linux-i686-2.4/socketmodule.o /tmp/Python-2.4/Modules/socketmodule.c: In function `socket_inet_ntop': /tmp/Python-2.4/Modules/socketmodule.c:3350: `INET_ADDRSTRLEN' undeclared (first use this function) /tmp/Python-2.4/Modules/socketmodule.c:3350: (Each undeclared identifier is reported only once /tmp/Python-2.4/Modules/socketmodule.c:3350: for each function it appears in.) /tmp/Python-2.4/Modules/socketmodule.c:3350: size of array `ip' has non-integer type ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 23:23 Message: Logged In: YES user_id=21627 Unfortunately, non of us has Solaris 7, so there is little chance that we are able to develop a patch. Can you provide one? ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-11 18:37 Message: Logged In: YES user_id=377412 I got: building '_socket' extension gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -fno-strict-aliasing -I. -I/opt/src/Python-2.4.1/./Include -I/usr/local/include -I/opt/src/Python-2.4.1/Include -I/opt/src/Python-2.4.1 -c /opt/src/Python-2.4.1/Modules/socketmodule.c -o build/temp.solaris-2.7-sun4u-2.4/socketmodule.o In file included from /opt/src/Python-2.4.1/Modules/socketmodule.c:315: /opt/src/Python-2.4.1/Modules/getnameinfo.c: In function `fake_getnameinfo': /opt/src/Python-2.4.1/Modules/getnameinfo.c:173: warning: implicit declaration of function `inet_ntop' /opt/src/Python-2.4.1/Modules/getnameinfo.c:174: warning: comparison between pointer and integer /opt/src/Python-2.4.1/Modules/getnameinfo.c:206: warning: comparison between pointer and integer /opt/src/Python-2.4.1/Modules/socketmodule.c: In function `socket_inet_ntop': /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: `INET_ADDRSTRLEN' undeclared (first use in this function) /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: (Each undeclared identifier is reported only once /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: for each function it appears in.) /opt/src/Python-2.4.1/Modules/socketmodule.c:3380: warning: assignment makes pointer from integer without a cast /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: warning: unused variable `ip' My users then see... # regtester.py ../ini/tester.ini main Traceback (most recent call last): File "./regtester.py", line 24, in ? import core_engine, core_event, core_action, logger File "/etsi/tst/regtester.1.0.0/scripts/core_engine.py", line 19, in ? import time, Queue, sched, copy, string, threading, asyncore, socket File "/usr/local/lib/python2.4/asyncore.py", line 51, in ? import socket File "/usr/local/lib/python2.4/socket.py", line 45, in ? import _socket ImportError: No module named _socket I try your work-around. Doesn't help. ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-11 18:00 Message: Logged In: YES user_id=377412 I have the same problem on Solaris 7. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-01-06 21:35 Message: Logged In: YES user_id=21627 There are already a few attempts to define ADDRSTRLEN if it is missing. Maybe we should just merge them into one, and define it if it is not defined, independent of the system. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-13 14:19 Message: Logged In: YES user_id=29957 How old a debian release is this? It sounds like it's a just a broken system header, and I'm not entirely convinced that adding this to the already-large collection of #ifdefs in socketmodule is a path to happiness. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 From noreply at sourceforge.net Mon Apr 11 23:25:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 23:25:17 2005 Subject: [ python-Bugs-1180997 ] lax error-checking in new-in-2.4 marshal stuff Message-ID: Bugs item #1180997, was opened at 2005-04-11 21:53 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180997&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None >Priority: 7 Submitted By: Michael Hudson (mwh) Assigned to: Martin v. L?wis (loewis) Summary: lax error-checking in new-in-2.4 marshal stuff Initial Comment: I realise one of the points of the TYPE_STRINGREF and so on stuff was efficiency, but: >>> marshal.loads('R') # TYPE_STRINGREF == 'R' Segmentation fault ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 23:25 Message: Logged In: YES user_id=21627 I agree, and will try to work on a patch before 2.4.2. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180997&group_id=5470 From noreply at sourceforge.net Mon Apr 11 23:42:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 11 23:42:55 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 17:14 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 >Status: Open Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-11 23:42 Message: Logged In: YES user_id=89016 OK, I'm reopening to bug report. I didn't manage to install pythondoc. cElementTree complains about: No such file or directory: './pyconfig.h'. Can you provide a simple Python file that fails when imported? ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-10 00:47 Message: Logged In: YES user_id=86307 Sorry to comment on a closed report, but perhaps this fix should not be limited only to cases where size is None. Today, I ran into a spurious syntax error when trying to import pythondoc (from http://effbot.org/downloads/pythondoc-2.1b3-20050325.zip). It turned out a \r was ending up in what looked to the parser like the middle of a line, presumably because a \n was dropped. Anyway, I applied the referenced patch to 2.4.1, except I left out the "size is None" condition, since I knew the tokenizer passes in a size param. With that change pythondoc import successfully. (Also, I just ran the test suite and nothing broke.) Since the size parameter is already documented as being passed to StreamReader.read (in codecs.py -- the HTML documentation needs to be updated), and since StreamReader.read says size is an approximate maximum, perhaps it's OK to read one extra byte. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-05 00:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Tue Apr 12 12:56:18 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 12 12:56:21 2005 Subject: [ python-Bugs-1180267 ] expanding platform module and making it work as it should Message-ID: Bugs item #1180267, was opened at 2005-04-10 19:44 Message generated for change (Comment added) made by nkour You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nikos Kouremenos (nkour) Assigned to: Nobody/Anonymous (nobody) Summary: expanding platform module and making it work as it should Initial Comment: platform.release() (which is supposed to return the Name of Windows) also does not work as it should in some versions of windows I tried (xp pro sp1). Luckily to print more than Windows (eg. print Windows XP or Windows 2000 etc) you can have a look at this http://www.brunningonline.net/simon/blog/archives/001168.html of Simon Brunning also only debian, mdk and redhat is scanned for GNU/Linux ?? why even bother then? I think that PSL is good but this module is has hell of limitations. At least dont' make anyone write this: import os distro_info = { 'Arch Linux': '/etc/arch-release', 'Aurox Linux': '/etc/aurox-release', 'Conectiva Linux': '/etc/conectiva-release', 'Debian GNU/Linux': '/etc/debian_release', 'Debian GNU/Linux': '/etc/debian_version', 'Fedora Linux': '/etc/fedora-release', 'Gentoo Linux': '/etc/gentoo-release', 'Mandrake Linux': '/etc/mandrake-release', 'Slackware Linux': '/etc/slackware-release', 'Slackware Linux': '/etc/slackware-version', 'Solaris/Sparc': '/etc/release', 'Sun JDS': '/etc/sun-release', 'Novell SUSE Linux': '/etc/SuSE-release', 'PLD Linux': '/etc/pld-release', 'SUSE Linux': '/etc/SuSE-release', 'Yellow Dog Linux': '/etc/yellowdog-release', # many distros use the /etc/redhat-release for compatibility # so Redhat is the last 'Redhat Linux': '/etc/redhat-release'} def get_os_info(): if os.name =='nt': win_version = { (1, 4, 0): "95", (1, 4, 10): "98", (1, 4, 90): "ME", (2, 4, 0): "NT", (2, 5, 0): "2000", (2, 5, 1): "XP" }[os.sys.getwindowsversion()[3], os.sys.getwindowsversion()[0], os.sys.getwindowsversion()[1]] return 'Windows' + ' ' + win_version elif os.name =='posix': executable = 'lsb_release' params = ' --id --codename --release --short' for path in os.environ['PATH'].split(':'): full_path_to_executable = os.path.join(path, executable) if os.path.exists(full_path_to_executable): command = executable + params child_stdin, child_stdout = os.popen2(command) output = child_stdout.readline().strip() child_stdout.close() child_stdin.close() return output # lsb_release executable not available, so parse files for distro in distro_info: path_to_file = distro_info[distro] if os.path.exists(path_to_file): file = open(path_to_file) text = file.read().strip() file.close() if path_to_file.endswith('version'): text = distro + ' ' + text return text print get_os_info() Thank you ---------------------------------------------------------------------- >Comment By: Nikos Kouremenos (nkour) Date: 2005-04-12 13:56 Message: Logged In: YES user_id=865368 ok I have it again in a friend of mine. he has win xp pro sp1 French edition and release() returns "" the Simon's trick will return to him XP which is correct!! ---------------------------------------------------------------------- Comment By: Nikos Kouremenos (nkour) Date: 2005-04-11 15:38 Message: Logged In: YES user_id=865368 ok nevermind the platform.release() part for windows name. it seems to work now.. :$ ---------------------------------------------------------------------- Comment By: Nikos Kouremenos (nkour) Date: 2005-04-10 19:46 Message: Logged In: YES user_id=865368 identation went to take a walk :( I'm sorry look here: http://nkour.blogspot.com/2005/03/python-script-to-detect-gnulinux.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 From noreply at sourceforge.net Tue Apr 12 18:49:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 12 18:49:54 2005 Subject: [ python-Bugs-1181619 ] Bad sys.executable value for bdist_wininst install script Message-ID: Bugs item #1181619, was opened at 2005-04-12 16:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181619&group_id=5470 Category: Distutils Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: follower (xfollower) Assigned to: Nobody/Anonymous (nobody) Summary: Bad sys.executable value for bdist_wininst install script Initial Comment: Description copied from: >From the Python docs, sys.executable is: executable A string giving the name of the executable binary for the Python interpreter, on systems where this makes sense. However, during the execution of a post-install script, this string actually resolves to the name of the binary installer! This name should resolve, I think to the name of the Python executable for which the installer is running (a value selectable at the start of the installation, if more than one Python is detected). Having this value available allows you to properly generate shortcuts with the proper full path to the python executable. I resorted to using sys.prefix+r'\python.exe', which will most likely work, but I'd rather see sys.executable give me a more sensible answer. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181619&group_id=5470 From noreply at sourceforge.net Tue Apr 12 21:40:38 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 12 21:40:42 2005 Subject: [ python-Bugs-1078245 ] Python2.4: building '_socket' extension fails with `INET_ADD Message-ID: Bugs item #1078245, was opened at 2004-12-03 05:16 Message generated for change (Comment added) made by ggarrett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Str?der (stroeder) Assigned to: Nobody/Anonymous (nobody) Summary: Python2.4: building '_socket' extension fails with `INET_ADD Initial Comment: HI! I'm trying to build Python2.4 on a rather old Debian machine. Building _socket fails (see below) although I tried to use configure --disable-ipv6 I've added #define INET_ADDRSTRLEN 16 to Modules/socketmodule.h which made it work. There were no problems compiling Python2.2 on the very same machine. Ciao, Michael. ------------------------------ snip ------------------------------ building '_socket' extension gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/tmp/Python-2.4/./Include -I/user/W3311/local/include -I/usr/local/include -I/tmp/Python-2.4/Include -I/tmp/Python-2.4 -c /tmp/Python-2.4/Modules/socketmodule.c -o build/t emp.linux-i686-2.4/socketmodule.o /tmp/Python-2.4/Modules/socketmodule.c: In function `socket_inet_ntop': /tmp/Python-2.4/Modules/socketmodule.c:3350: `INET_ADDRSTRLEN' undeclared (first use this function) /tmp/Python-2.4/Modules/socketmodule.c:3350: (Each undeclared identifier is reported only once /tmp/Python-2.4/Modules/socketmodule.c:3350: for each function it appears in.) /tmp/Python-2.4/Modules/socketmodule.c:3350: size of array `ip' has non-integer type ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-12 14:40 Message: Logged In: YES user_id=377412 I'm a SysAdmin. My users (programmers) who want this version of Python claim to have put this fix in place and it works (on a copy I'm sure is squirreled away in someone's home directory). Perhaps the account I'm using doesn't have something it needs in LD_LIBRARY_PATH, etc.? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 16:23 Message: Logged In: YES user_id=21627 Unfortunately, non of us has Solaris 7, so there is little chance that we are able to develop a patch. Can you provide one? ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-11 11:37 Message: Logged In: YES user_id=377412 I got: building '_socket' extension gcc -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -fno-strict-aliasing -I. -I/opt/src/Python-2.4.1/./Include -I/usr/local/include -I/opt/src/Python-2.4.1/Include -I/opt/src/Python-2.4.1 -c /opt/src/Python-2.4.1/Modules/socketmodule.c -o build/temp.solaris-2.7-sun4u-2.4/socketmodule.o In file included from /opt/src/Python-2.4.1/Modules/socketmodule.c:315: /opt/src/Python-2.4.1/Modules/getnameinfo.c: In function `fake_getnameinfo': /opt/src/Python-2.4.1/Modules/getnameinfo.c:173: warning: implicit declaration of function `inet_ntop' /opt/src/Python-2.4.1/Modules/getnameinfo.c:174: warning: comparison between pointer and integer /opt/src/Python-2.4.1/Modules/getnameinfo.c:206: warning: comparison between pointer and integer /opt/src/Python-2.4.1/Modules/socketmodule.c: In function `socket_inet_ntop': /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: `INET_ADDRSTRLEN' undeclared (first use in this function) /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: (Each undeclared identifier is reported only once /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: error: for each function it appears in.) /opt/src/Python-2.4.1/Modules/socketmodule.c:3380: warning: assignment makes pointer from integer without a cast /opt/src/Python-2.4.1/Modules/socketmodule.c:3350: warning: unused variable `ip' My users then see... # regtester.py ../ini/tester.ini main Traceback (most recent call last): File "./regtester.py", line 24, in ? import core_engine, core_event, core_action, logger File "/etsi/tst/regtester.1.0.0/scripts/core_engine.py", line 19, in ? import time, Queue, sched, copy, string, threading, asyncore, socket File "/usr/local/lib/python2.4/asyncore.py", line 51, in ? import socket File "/usr/local/lib/python2.4/socket.py", line 45, in ? import _socket ImportError: No module named _socket I try your work-around. Doesn't help. ---------------------------------------------------------------------- Comment By: Garry J. Garrett (ggarrett) Date: 2005-04-11 11:00 Message: Logged In: YES user_id=377412 I have the same problem on Solaris 7. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-01-06 14:35 Message: Logged In: YES user_id=21627 There are already a few attempts to define ADDRSTRLEN if it is missing. Maybe we should just merge them into one, and define it if it is not defined, independent of the system. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2004-12-13 07:19 Message: Logged In: YES user_id=29957 How old a debian release is this? It sounds like it's a just a broken system header, and I'm not entirely convinced that adding this to the already-large collection of #ifdefs in socketmodule is a path to happiness. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1078245&group_id=5470 From noreply at sourceforge.net Wed Apr 13 04:12:45 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 13 04:12:48 2005 Subject: [ python-Bugs-1181939 ] asyncore.loop() documentation Message-ID: Bugs item #1181939, was opened at 2005-04-13 11:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181939&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Graham (gumleef) Assigned to: Nobody/Anonymous (nobody) Summary: asyncore.loop() documentation Initial Comment: Document at http://docs.python.org/lib/module-asyncore.html says that the map dictionary is updated in the asyncore class's __init__() method (and to make sure you extend this if you wish to retain this behaviour). This is not the case. The map dictionary is updated by asyncore.create_socket(). asyncore does not need to be extended to retain the mapping behaviour the documentation refers to. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181939&group_id=5470 From noreply at sourceforge.net Wed Apr 13 12:05:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 13 12:05:40 2005 Subject: [ python-Bugs-1180997 ] lax error-checking in new-in-2.4 marshal stuff Message-ID: Bugs item #1180997, was opened at 2005-04-11 20:53 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180997&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 7 Submitted By: Michael Hudson (mwh) Assigned to: Martin v. L?wis (loewis) Summary: lax error-checking in new-in-2.4 marshal stuff Initial Comment: I realise one of the points of the TYPE_STRINGREF and so on stuff was efficiency, but: >>> marshal.loads('R') # TYPE_STRINGREF == 'R' Segmentation fault ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-13 11:05 Message: Logged In: YES user_id=6656 While I have your attention: >>> marshal.loads("", 1) Segmentation fault This is the guilty line: if (!PyArg_ParseTuple(args, "s#|i:loads", &s, &n)) there's no pointer corresponding to the optional integer argument. I'd just fix this, but I have too many local changes to make it easy :( ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 22:25 Message: Logged In: YES user_id=21627 I agree, and will try to work on a patch before 2.4.2. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180997&group_id=5470 From noreply at sourceforge.net Wed Apr 13 13:02:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 13 13:03:00 2005 Subject: [ python-Feature Requests-1182143 ] making builtin exceptions more informative Message-ID: Feature Requests item #1182143, was opened at 2005-04-13 11:02 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1182143&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sebastien de Menten (sdementen) Assigned to: Nobody/Anonymous (nobody) Summary: making builtin exceptions more informative Initial Comment: Using builtin exception information is tricky as it consists of: a) the type of exception (easily accessible) b) the args attribute = a 1 element tuple with a string 1st example: try: print foo except NameError, e: print e.args symbol = e.args[0][17:-16] print symbols ==> ("NameError: name 'foo' is not defined", ) foo It would be nicer to have: e.args = ("NameError: name 'foo' is not defined", "foo") The first element being the current string for backward compatibilty. ============================= 2nd example: try: (4).foo except NameError, e: print e.args ==> ("'int' object has no attribute 'foo'",) It would be nicer to have: e.args = ("'int' object has no attribute 'foo'", 4, "foo") Again, the first element being the current string for backward compatibilty. ============================= Moreover, in the documentation about Exception, I read """Warning: Messages to exceptions are not part of the Python API. Their contents may change from one version of Python to the next without warning and should not be relied on by code which will run under multiple versions of the interpreter. """ So even args could not be relied upon ! But it also means that there is no need to be backward compatible (I am playing devil's advocate, backward compatibility is important !) Seb ps: There may be problems (that I am not aware) with a) an exception keeping references to other objects b) C API that can throw only exceptions with strings c) a specific advantage of having strings only in builtin exceptions ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1182143&group_id=5470 From noreply at sourceforge.net Wed Apr 13 13:12:36 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 13 13:12:39 2005 Subject: [ python-Bugs-1180267 ] expanding platform module and making it work as it should Message-ID: Bugs item #1180267, was opened at 2005-04-10 19:44 Message generated for change (Comment added) made by nkour You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nikos Kouremenos (nkour) Assigned to: Nobody/Anonymous (nobody) Summary: expanding platform module and making it work as it should Initial Comment: platform.release() (which is supposed to return the Name of Windows) also does not work as it should in some versions of windows I tried (xp pro sp1). Luckily to print more than Windows (eg. print Windows XP or Windows 2000 etc) you can have a look at this http://www.brunningonline.net/simon/blog/archives/001168.html of Simon Brunning also only debian, mdk and redhat is scanned for GNU/Linux ?? why even bother then? I think that PSL is good but this module is has hell of limitations. At least dont' make anyone write this: import os distro_info = { 'Arch Linux': '/etc/arch-release', 'Aurox Linux': '/etc/aurox-release', 'Conectiva Linux': '/etc/conectiva-release', 'Debian GNU/Linux': '/etc/debian_release', 'Debian GNU/Linux': '/etc/debian_version', 'Fedora Linux': '/etc/fedora-release', 'Gentoo Linux': '/etc/gentoo-release', 'Mandrake Linux': '/etc/mandrake-release', 'Slackware Linux': '/etc/slackware-release', 'Slackware Linux': '/etc/slackware-version', 'Solaris/Sparc': '/etc/release', 'Sun JDS': '/etc/sun-release', 'Novell SUSE Linux': '/etc/SuSE-release', 'PLD Linux': '/etc/pld-release', 'SUSE Linux': '/etc/SuSE-release', 'Yellow Dog Linux': '/etc/yellowdog-release', # many distros use the /etc/redhat-release for compatibility # so Redhat is the last 'Redhat Linux': '/etc/redhat-release'} def get_os_info(): if os.name =='nt': win_version = { (1, 4, 0): "95", (1, 4, 10): "98", (1, 4, 90): "ME", (2, 4, 0): "NT", (2, 5, 0): "2000", (2, 5, 1): "XP" }[os.sys.getwindowsversion()[3], os.sys.getwindowsversion()[0], os.sys.getwindowsversion()[1]] return 'Windows' + ' ' + win_version elif os.name =='posix': executable = 'lsb_release' params = ' --id --codename --release --short' for path in os.environ['PATH'].split(':'): full_path_to_executable = os.path.join(path, executable) if os.path.exists(full_path_to_executable): command = executable + params child_stdin, child_stdout = os.popen2(command) output = child_stdout.readline().strip() child_stdout.close() child_stdin.close() return output # lsb_release executable not available, so parse files for distro in distro_info: path_to_file = distro_info[distro] if os.path.exists(path_to_file): file = open(path_to_file) text = file.read().strip() file.close() if path_to_file.endswith('version'): text = distro + ' ' + text return text print get_os_info() Thank you ---------------------------------------------------------------------- >Comment By: Nikos Kouremenos (nkour) Date: 2005-04-13 14:12 Message: Logged In: YES user_id=865368 A friend of mine has a french win xp pro. your way says to him WP platform.release() returns "" to him he has used antispam and such "tools" so he must have accidentaly removed the place where release() seems to look. So release() doesn't do for windows what you do. I also noticed that On a windows exploer help --> about will give Microsoft Windows 5.1 (more info) it won't try the magic that release() [that fail] simon's code still works there as charm. so if release() returns "" I try to do that other code. I would like to see that included though [or a better code for release() to detect the correct stuff] along with the more linux distros and the usage of lsb_release Comments? ---------------------------------------------------------------------- Comment By: Nikos Kouremenos (nkour) Date: 2005-04-12 13:56 Message: Logged In: YES user_id=865368 ok I have it again in a friend of mine. he has win xp pro sp1 French edition and release() returns "" the Simon's trick will return to him XP which is correct!! ---------------------------------------------------------------------- Comment By: Nikos Kouremenos (nkour) Date: 2005-04-11 15:38 Message: Logged In: YES user_id=865368 ok nevermind the platform.release() part for windows name. it seems to work now.. :$ ---------------------------------------------------------------------- Comment By: Nikos Kouremenos (nkour) Date: 2005-04-10 19:46 Message: Logged In: YES user_id=865368 identation went to take a walk :( I'm sorry look here: http://nkour.blogspot.com/2005/03/python-script-to-detect-gnulinux.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180267&group_id=5470 From noreply at sourceforge.net Thu Apr 14 00:54:55 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 00:54:57 2005 Subject: [ python-Bugs-1182603 ] re.escape(s) prints wrong for chr(0) Message-ID: Bugs item #1182603, was opened at 2005-04-13 17:54 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182603&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Jacobson (nickjacobson) Assigned to: Nobody/Anonymous (nobody) Summary: re.escape(s) prints wrong for chr(0) Initial Comment: >>> import re >>> s = chr(0) >>> s '\x00' >>> re.escape(s) '\000' I believe it should print: '\\x00' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182603&group_id=5470 From noreply at sourceforge.net Thu Apr 14 01:21:32 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 01:21:35 2005 Subject: [ python-Bugs-1182614 ] dir() does not include _ Message-ID: Bugs item #1182614, was opened at 2005-04-13 18:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182614&group_id=5470 Category: None Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Jacobson (nickjacobson) Assigned to: Nobody/Anonymous (nobody) Summary: dir() does not include _ Initial Comment: At the interpreter prompt, dir() does not include the variable _ >>> dir() ['__builtins__', '__doc__', '__name__'] >>> x = 5 >>> dir() ['__builtins__', '__doc__', '__name__', 'x'] >>> _ ['__builtins__', '__doc__', '__name__', 'x'] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182614&group_id=5470 From noreply at sourceforge.net Thu Apr 14 10:10:04 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 10:10:07 2005 Subject: [ python-Bugs-1182788 ] ZipFile __del__/close problem with longint/long files Message-ID: Bugs item #1182788, was opened at 2005-04-14 10:10 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182788&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Robert Kiendl (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile __del__/close problem with longint/long files Initial Comment: >From a user (py2.3.4 based app) I got only this small trace: I cannot reproduce it easily: --- sync complete: 6 files (0+6), 3488210kB copied. --- 1795.9 Seconds (100.0% CPU Time) Exception exceptions.OverflowError: 'long int too large to convert to int' in > ignored ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182788&group_id=5470 From noreply at sourceforge.net Thu Apr 14 11:56:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 11:57:02 2005 Subject: [ python-Bugs-1180147 ] test_posix fails on cygwin Message-ID: Bugs item #1180147, was opened at 2005-04-10 13:10 Message generated for change (Comment added) made by wist You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180147&group_id=5470 Category: None Group: Python 2.3 Status: Open >Resolution: Invalid Priority: 5 Submitted By: Henrik Wist (wist) Assigned to: Nobody/Anonymous (nobody) Summary: test_posix fails on cygwin Initial Comment: $ python test/test_posix.py testNoArgFunctions (__main__.PosixTester) ... ERROR test_access (__main__.PosixTester) ... ok test_chdir (__main__.PosixTester) ... ok test_dup (__main__.PosixTester) ... ok test_dup2 (__main__.PosixTester) ... ok test_fdopen (__main__.PosixTester) ... ok test_fstat (__main__.PosixTester) ... ok test_fstatvfs (__main__.PosixTester) ... ok test_ftruncate (__main__.PosixTester) ... ok test_lsdir (__main__.PosixTester) ... ok test_pipe (__main__.PosixTester) ... ok test_stat (__main__.PosixTester) ... ok test_statvfs (__main__.PosixTester) ... ok test_strerror (__main__.PosixTester) ... ok test_tempnam (__main__.PosixTester) ... ok test_tmpfile (__main__.PosixTester) ... ok test_umask (__main__.PosixTester) ... ok test_utime (__main__.PosixTester) ... ok ====================================================================== ERROR: testNoArgFunctions (__main__.PosixTester) ---------------------------------------------------------------------- Traceback (most recent call last): File "test/test_posix.py", line 40, in testNoArgFunctions posix_func() OSError: [Errno 22] Invalid argument ---------------------------------------------------------------------- Ran 18 tests in 0.038s FAILED (errors=1) Traceback (most recent call last): File "test/test_posix.py", line 159, in ? test_main() File "test/test_posix.py", line 156, in test_main test_support.run_unittest(PosixTester) File "/usr/lib/python2.3/test/test_support.py", line 262, in run_unittest run_suite(suite, testclass) File "/usr/lib/python2.3/test/test_support.py", line 247, in run_suite raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "test/test_posix.py", line 40, in testNoArgFunctions posix_func() OSError: [Errno 22] Invalid argument This is with cygwin 1.5.12-1 and python 2.3.4-2. ---------------------------------------------------------------------- >Comment By: Henrik Wist (wist) Date: 2005-04-14 11:56 Message: Logged In: YES user_id=1256464 Strangely enough, this works when deleting /etc/group or creating it with either local groups (mkgroup -l) or domain groups (mkgroup -d). I guess it is more a cygwin problem, then. ---------------------------------------------------------------------- Comment By: Henrik Wist (wist) Date: 2005-04-10 13:12 Message: Logged In: YES user_id=1256464 And, I forgot, this is with WinXP SP2 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180147&group_id=5470 From noreply at sourceforge.net Thu Apr 14 16:26:38 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 16:26:40 2005 Subject: [ python-Bugs-1099364 ] raw_input() displays wrong unicode prompt Message-ID: Bugs item #1099364, was opened at 2005-01-10 11:33 Message generated for change (Comment added) made by prikryl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1099364&group_id=5470 Category: XML Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Petr Prikryl (prikryl) Assigned to: Nobody/Anonymous (nobody) Summary: raw_input() displays wrong unicode prompt Initial Comment: I have observed a problem when running Python 2.4, Windows version (python-2.4.msi) and using raw_input() with unicode prompt string in a console program (ran in the DOS window). I do use the following sitecustomize.py file to set the default encoding in the English Windows 2000 Server: sitecustomize.py ================================= import sys sys.setdefaultencoding('cp1250') ================================= test.py ================================= # -*- coding: cp1250 -*- s = u'string with accented letters (different than this)' print s # OK val = raw_input(s) # s displayed differently (wrong) ================================= See the test.png (captured from screen) and the test.py for the used string -- inside the attached zip file. The "type test.py" (result visible on the captured screen) displays the string definition also wrongly, because the DOS window uses different encoding than cp1250. The print command prints the string correctly, converting the internal unicode string to the encoding that the is defined by the output environment. However, the raw_input() probably does convert the unicode string to the cp1250 and does not do the same (more clever) thing that the print does. I did not use the unicode in older Python (2.3.4), so I do not know what was the behaviour earlier. Could you confirm the bug? Sorry if the bug is well known. Petr ---------------------------------------------------------------------- >Comment By: Petr Prikryl (prikryl) Date: 2005-04-14 16:26 Message: Logged In: YES user_id=771873 New observation: sys.stdout.write(s) behaves visually on the screen exactly as the raw_input(s) does. So, print does something more when displaying on the screen... Petr ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1099364&group_id=5470 From noreply at sourceforge.net Thu Apr 14 16:31:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 16:32:02 2005 Subject: [ python-Bugs-1099364 ] raw_input() displays wrong unicode prompt Message-ID: Bugs item #1099364, was opened at 2005-01-10 11:33 Message generated for change (Settings changed) made by prikryl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1099364&group_id=5470 Category: XML Group: Python 2.4 Status: Open Resolution: None >Priority: 6 Submitted By: Petr Prikryl (prikryl) Assigned to: Nobody/Anonymous (nobody) Summary: raw_input() displays wrong unicode prompt Initial Comment: I have observed a problem when running Python 2.4, Windows version (python-2.4.msi) and using raw_input() with unicode prompt string in a console program (ran in the DOS window). I do use the following sitecustomize.py file to set the default encoding in the English Windows 2000 Server: sitecustomize.py ================================= import sys sys.setdefaultencoding('cp1250') ================================= test.py ================================= # -*- coding: cp1250 -*- s = u'string with accented letters (different than this)' print s # OK val = raw_input(s) # s displayed differently (wrong) ================================= See the test.png (captured from screen) and the test.py for the used string -- inside the attached zip file. The "type test.py" (result visible on the captured screen) displays the string definition also wrongly, because the DOS window uses different encoding than cp1250. The print command prints the string correctly, converting the internal unicode string to the encoding that the is defined by the output environment. However, the raw_input() probably does convert the unicode string to the cp1250 and does not do the same (more clever) thing that the print does. I did not use the unicode in older Python (2.3.4), so I do not know what was the behaviour earlier. Could you confirm the bug? Sorry if the bug is well known. Petr ---------------------------------------------------------------------- Comment By: Petr Prikryl (prikryl) Date: 2005-04-14 16:26 Message: Logged In: YES user_id=771873 New observation: sys.stdout.write(s) behaves visually on the screen exactly as the raw_input(s) does. So, print does something more when displaying on the screen... Petr ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1099364&group_id=5470 From noreply at sourceforge.net Thu Apr 14 16:34:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 16:34:16 2005 Subject: [ python-Bugs-1099364 ] raw_input() displays wrong unicode prompt Message-ID: Bugs item #1099364, was opened at 2005-01-10 11:33 Message generated for change (Comment added) made by prikryl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1099364&group_id=5470 Category: XML Group: Python 2.4 Status: Open Resolution: None Priority: 6 Submitted By: Petr Prikryl (prikryl) Assigned to: Nobody/Anonymous (nobody) Summary: raw_input() displays wrong unicode prompt Initial Comment: I have observed a problem when running Python 2.4, Windows version (python-2.4.msi) and using raw_input() with unicode prompt string in a console program (ran in the DOS window). I do use the following sitecustomize.py file to set the default encoding in the English Windows 2000 Server: sitecustomize.py ================================= import sys sys.setdefaultencoding('cp1250') ================================= test.py ================================= # -*- coding: cp1250 -*- s = u'string with accented letters (different than this)' print s # OK val = raw_input(s) # s displayed differently (wrong) ================================= See the test.png (captured from screen) and the test.py for the used string -- inside the attached zip file. The "type test.py" (result visible on the captured screen) displays the string definition also wrongly, because the DOS window uses different encoding than cp1250. The print command prints the string correctly, converting the internal unicode string to the encoding that the is defined by the output environment. However, the raw_input() probably does convert the unicode string to the cp1250 and does not do the same (more clever) thing that the print does. I did not use the unicode in older Python (2.3.4), so I do not know what was the behaviour earlier. Could you confirm the bug? Sorry if the bug is well known. Petr ---------------------------------------------------------------------- >Comment By: Petr Prikryl (prikryl) Date: 2005-04-14 16:34 Message: Logged In: YES user_id=771873 Python 2.4.1 for Windows behaves the same way. Petr ---------------------------------------------------------------------- Comment By: Petr Prikryl (prikryl) Date: 2005-04-14 16:26 Message: Logged In: YES user_id=771873 New observation: sys.stdout.write(s) behaves visually on the screen exactly as the raw_input(s) does. So, print does something more when displaying on the screen... Petr ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1099364&group_id=5470 From noreply at sourceforge.net Thu Apr 14 17:30:39 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 17:30:42 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by zosh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: zosh (zosh) Date: 2005-04-14 17:30 Message: Logged In: YES user_id=1254741 I must assume that according to the snippet of unistd.h I provided in an earlier comment. Is it possible that setting one -D option clears another one like _SGIAPI ? Tell me please if you would like me perform more tests on my system. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 23:21 Message: Logged In: YES user_id=21627 Hmm. You say "because cc should report a non-existent setgroups otherwise"; then you bring an error message '"setgroups" is undefined". Doesn't this imply that _SGIAPI (not _SIGAPI) is indeed not defined, in the case of Python, as well as the case of the various -D options? ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-11 14:29 Message: Logged In: YES user_id=1254741 I believe _SIGAPI is defined beacuse cc should report a non-existent "setgroups" otherwise. Compiling my test program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 resulted in a failure: cc-1020 cc: ERROR File = t.c, Line = 5 The identifier "setgroups" is undefined. void *p = setgroups; ^ ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-09 12:33 Message: Logged In: YES user_id=21627 Why do you believe that _SGIAPI is defined? Please try compiling your program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-08 09:51 Message: Logged In: YES user_id=1254741 My test program that compiled correctly with "cc -c -g": #include int main () { void *p = setgroups; return 0; } The definition of setgroups in : #if _SGIAPI ... #if defined(_BSD_COMPAT) extern int setgroups(int, int *); #else /* !_BSD_COMPAT */ extern int setgroups(int, const gid_t *); #endif /* _BSD_COMPAT */ ... #endif /* _SGIAPI */ So it should catch one of these defines. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Thu Apr 14 20:16:50 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 20:16:53 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2005-04-14 20:16 Message: Logged In: YES user_id=21627 Please try to find out whether _SGIAPI is defined anywhere in /usr/include/* or /usr/include/*/*; most likely, some of the -D defines suppress the definition of _SGIAPI. If so, a solution might be to define _SGIAPI in addition to the other defines; see the definition of _GNU_SOURCE, _BSD_VISIBLE etc in configure.in. However, it would be good to find some definition for _SGIAPI - perhaps we are supposed to define _SGI_SOURCE, or _SGI_GIMME_EVERYTHING :-) ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-14 17:30 Message: Logged In: YES user_id=1254741 I must assume that according to the snippet of unistd.h I provided in an earlier comment. Is it possible that setting one -D option clears another one like _SGIAPI ? Tell me please if you would like me perform more tests on my system. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 23:21 Message: Logged In: YES user_id=21627 Hmm. You say "because cc should report a non-existent setgroups otherwise"; then you bring an error message '"setgroups" is undefined". Doesn't this imply that _SGIAPI (not _SIGAPI) is indeed not defined, in the case of Python, as well as the case of the various -D options? ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-11 14:29 Message: Logged In: YES user_id=1254741 I believe _SIGAPI is defined beacuse cc should report a non-existent "setgroups" otherwise. Compiling my test program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 resulted in a failure: cc-1020 cc: ERROR File = t.c, Line = 5 The identifier "setgroups" is undefined. void *p = setgroups; ^ ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-09 12:33 Message: Logged In: YES user_id=21627 Why do you believe that _SGIAPI is defined? Please try compiling your program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-08 09:51 Message: Logged In: YES user_id=1254741 My test program that compiled correctly with "cc -c -g": #include int main () { void *p = setgroups; return 0; } The definition of setgroups in : #if _SGIAPI ... #if defined(_BSD_COMPAT) extern int setgroups(int, int *); #else /* !_BSD_COMPAT */ extern int setgroups(int, const gid_t *); #endif /* _BSD_COMPAT */ ... #endif /* _SGIAPI */ So it should catch one of these defines. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Thu Apr 14 21:26:22 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 21:26:24 2005 Subject: [ python-Bugs-1180160 ] subprocess.Popen fails with closed stdout Message-ID: Bugs item #1180160, was opened at 2005-04-10 13:41 Message generated for change (Comment added) made by astrand You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180160&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: neuhauser (neuhauser) >Assigned to: Peter ?strand (astrand) Summary: subprocess.Popen fails with closed stdout Initial Comment: I have a piece of code that gets run in a script that has its stdout closed: import sys sys.stdout = sys.stderr c = subprocess.Popen (..., stdin = subprocess.PIPE, stdout = subprocess.PIPE, stderr = subprocess.STDOUT) and this is what I get: Sending SVNR/permissions Transmitting file data .svn: Commit failed (details follow): svn: 'pre-commit' hook failed with error output: Traceback (most recent call last): (...) File ".../__init__.py", line 40, in run stderr = subprocess.STDOUT) File "/usr/local/lib/python2.4/subprocess.py", line 554, in __init__ errread, errwrite) File "/usr/local/lib/python2.4/subprocess.py", line 986, in _execute_child raise child_exception OSError: [Errno 2] No such file or directory This is the child_traceback: File "/usr/local/lib/python2.4/subprocess.py", line 955, in _execute_child File "/usr/local/lib/python2.4/os.py", line 341, in execvp File "/usr/local/lib/python2.4/os.py", line 379, in _execvpe func(fullname, *argrest) OSError: [Errno 2] No such file or directory http://docs.python.org/lib/node230.html claims that "PIPE indicates that a new pipe to the child should be created" which doesn't seem to be the case. Subversion code that runs the script can be seen at http://svn.collab.net/viewcvs/svn/trunk/subversion/libsvn_repos/hooks.c (run_hook_cmd()). ---------------------------------------------------------------------- >Comment By: Peter ?strand (astrand) Date: 2005-04-14 21:26 Message: Logged In: YES user_id=344921 I can't se anything in your report that indicates that a closed stdout is the problem. execvp() says "No such file or directory", which means that the file you are trying to execute doesn't exist. Do you have any reason to believe that this error message is not true? It would help if you could give me a working *Python* example, indicating the problem. The source http://svn.collab.net/viewcvs/svn/trunk/subversion/libsvn_repos/hooks.c is written in C, not Python... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180160&group_id=5470 From noreply at sourceforge.net Thu Apr 14 21:32:17 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 21:32:19 2005 Subject: [ python-Bugs-1183274 ] Minor error in tutorial Message-ID: Bugs item #1183274, was opened at 2005-04-14 22:32 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183274&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Konrads Smelkovs (konrads-) Assigned to: Nobody/Anonymous (nobody) Summary: Minor error in tutorial Initial Comment: http://www.python.org/doc/current/tut/node11.html#SECTION00111100000000000000000 says: sine_table = dict((x, sin(x*pi/180)) for x in range(0, 91)) should be: sine_table = dict((x, sin(x*pi/180) for x in range(0, 91)) there is one too many brace after 180) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183274&group_id=5470 From noreply at sourceforge.net Thu Apr 14 21:50:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 21:50:49 2005 Subject: [ python-Bugs-1180237 ] Python keeps file references after calling close methode Message-ID: Bugs item #1180237, was opened at 2005-04-10 10:20 Message generated for change (Comment added) made by jepler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180237&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Eelco (eternia) Assigned to: Nobody/Anonymous (nobody) Summary: Python keeps file references after calling close methode Initial Comment: I found this bug using a python script that: - first mounts a partition (os.system("mount") etc) - change a few files on this partition (f = open (); f.write; f.close) - umounts the partition (os.system("umount") etc) Strangely, the umount didn't work because of a filesystem busy error. Using fuser and lsof i traced this being busy back to the script itself. This is strange behavior because after changing the files on the mounted partition the close method was called which should close all references to the file on the partition. Finally the solution was to do f = 0. So if python has closed a file on a mount a open reference to that file will keep to exist until the script has ended or until the file object is nullified. ---------------------------------------------------------------------- Comment By: Jeff Epler (jepler) Date: 2005-04-14 14:50 Message: Logged In: YES user_id=2772 I tried but can't reproduce this problem. After mounting a filesystem on /mnt/tmp, I ran the following in an interactive session: >>> f = open("/mnt/tmp/x", "w") >>> f.write("testing") >>> f.close() >>> import os; os.system("umount /mnt/tmp") 0 Did you write 'f.close()' or 'f.close'? The former closes the file, but the latter is a statement which has no effect. When you later executed 'f=0', the reference count of the-old-value-of-f dropped to 0 and the file was close()d impliclty. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1180237&group_id=5470 From noreply at sourceforge.net Thu Apr 14 22:01:11 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 22:01:14 2005 Subject: [ python-Bugs-1183274 ] Minor error in tutorial Message-ID: Bugs item #1183274, was opened at 2005-04-14 21:32 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183274&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Konrads Smelkovs (konrads-) Assigned to: Nobody/Anonymous (nobody) Summary: Minor error in tutorial Initial Comment: http://www.python.org/doc/current/tut/node11.html#SECTION00111100000000000000000 says: sine_table = dict((x, sin(x*pi/180)) for x in range(0, 91)) should be: sine_table = dict((x, sin(x*pi/180) for x in range(0, 91)) there is one too many brace after 180) ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-14 22:01 Message: Logged In: YES user_id=89016 The code is OK (this is a dictionary constructor consuming generator expression which generates (key, value) pairs. Wrapping the line should make it a little clearer: sine_table = dict( (x, sin(x*pi/180)) for x in range(0, 91) ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183274&group_id=5470 From noreply at sourceforge.net Thu Apr 14 22:17:26 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 22:17:29 2005 Subject: [ python-Bugs-1182788 ] ZipFile __del__/close problem with longint/long files Message-ID: Bugs item #1182788, was opened at 2005-04-14 03:10 Message generated for change (Comment added) made by alanmcintyre You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182788&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Robert Kiendl (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile __del__/close problem with longint/long files Initial Comment: >From a user (py2.3.4 based app) I got only this small trace: I cannot reproduce it easily: --- sync complete: 6 files (0+6), 3488210kB copied. --- 1795.9 Seconds (100.0% CPU Time) Exception exceptions.OverflowError: 'long int too large to convert to int' in > ignored ---------------------------------------------------------------------- Comment By: Alan McIntyre (ESRG) (alanmcintyre) Date: 2005-04-14 15:17 Message: Logged In: YES user_id=1115903 How do you reproduce it? I have seen that exception pop up when trying to add a file >2GB to an archive (although in that case it occurred during the call to ZipFile.write rather than __del__). It looks to me like writing the end-of-zip-archive record - during close(), which is called by __del__() - would give this exception if the contents of the archive were > 2GB, but I haven't tried this out to be sure. If that's what's causing the problem, then it would appear to me that it's just a limitation of the ZIP file format, but maybe the ZipFile object could detect this and raise a less cryptic exception. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182788&group_id=5470 From noreply at sourceforge.net Thu Apr 14 22:30:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 22:30:56 2005 Subject: [ python-Bugs-1182788 ] ZipFile __del__/close problem with longint/long files Message-ID: Bugs item #1182788, was opened at 2005-04-14 03:10 Message generated for change (Comment added) made by alanmcintyre You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182788&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Robert Kiendl (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile __del__/close problem with longint/long files Initial Comment: >From a user (py2.3.4 based app) I got only this small trace: I cannot reproduce it easily: --- sync complete: 6 files (0+6), 3488210kB copied. --- 1795.9 Seconds (100.0% CPU Time) Exception exceptions.OverflowError: 'long int too large to convert to int' in > ignored ---------------------------------------------------------------------- Comment By: Alan McIntyre (ESRG) (alanmcintyre) Date: 2005-04-14 15:30 Message: Logged In: YES user_id=1115903 Oh, silly me, I didn't notice this: "3488210kB copied," which is of course >2GB (don't know if your number indicates pre- or post-compression though). I'm assuming this means that you're bumping into the limits of the zip file format. Since this sort of thing happens occasionally in some applications I maintain, I will look at making a patch for ZipFile that will at least tell you when you exceed file format limits (either individual file size or total archive file size). ---------------------------------------------------------------------- Comment By: Alan McIntyre (ESRG) (alanmcintyre) Date: 2005-04-14 15:17 Message: Logged In: YES user_id=1115903 How do you reproduce it? I have seen that exception pop up when trying to add a file >2GB to an archive (although in that case it occurred during the call to ZipFile.write rather than __del__). It looks to me like writing the end-of-zip-archive record - during close(), which is called by __del__() - would give this exception if the contents of the archive were > 2GB, but I haven't tried this out to be sure. If that's what's causing the problem, then it would appear to me that it's just a limitation of the ZIP file format, but maybe the ZipFile object could detect this and raise a less cryptic exception. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182788&group_id=5470 From noreply at sourceforge.net Thu Apr 14 22:58:00 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 22:58:02 2005 Subject: [ python-Bugs-1179168 ] IDLE bug - changing shortcuts Message-ID: Bugs item #1179168, was opened at 2005-04-08 06:54 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179168&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Przemysław Gocyła (bld_d16) >Assigned to: Kurt B. Kaiser (kbk) Summary: IDLE bug - changing shortcuts Initial Comment: On Python 2.4.1 on WinXP I have changed keyboard shortcut settings for indent/unindent. New shorcut was Tab/Shift+Tab. When I did it IDLE hanged up and I could not to launch it again. I have tried everything. Uninstall whole Python and then reinstall (tried both version 2.3 and 2.4). It just looks like from the change of shortcuts IDLE don't works. I tried to: import idlelib.PyShell idlelib.PyShell.main() and result with bug are in attachment Where is the information about IDLE shortcuts keeped ? Why IDLE remember this even after reinstall ? How to fix it up? Best regards, Przemek ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2005-04-14 15:58 Message: Logged In: YES user_id=149084 Your user keyset config file on Windows is at Documents and Settings\\.idlerc\config-keys.cfg This file doesn't get removed when Python is uninstalled. Open this file with a text editor and find the entry. Change it to This is a bug in IDLE. (normally people don't try to re-assign the tab key, so it got missed) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1179168&group_id=5470 From noreply at sourceforge.net Thu Apr 14 23:00:00 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 23:00:11 2005 Subject: [ python-Bugs-1114776 ] copy.py bug Message-ID: Bugs item #1114776, was opened at 2005-02-02 09:43 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1114776&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: Fixed Priority: 9 Submitted By: Vincenzo Di Somma (disommav) Assigned to: Anthony Baxter (anthonybaxter) Summary: copy.py bug Initial Comment: upgrading from python 2.3.4 to 2.3.5rc1 I got this error on a module that was working correctly File "../Field.py", line 203, in copy properties = deepcopy(cdict) File "/usr/lib/python2.3/copy.py", line 191, in deepcopy y = copier(x, memo) File "/usr/lib/python2.3/copy.py", line 285, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.3/copy.py", line 200, in deepcopy copier = _getspecial(cls, "__deepcopy__") File "/usr/lib/python2.3/copy.py", line 66, in _getspecial for basecls in cls.__mro__: AttributeError: __mro__ ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2005-04-14 16:00 Message: Logged In: YES user_id=149084 Should this be closed? ---------------------------------------------------------------------- Comment By: Alex Martelli (aleax) Date: 2005-02-06 02:57 Message: Logged In: YES user_id=60314 Fixed (using inspect.getmro instead of direct __mro__ access) -- added unittest for that thanks to John Lenton's suggestion on c.l.py ---------------------------------------------------------------------- Comment By: Alex Martelli (aleax) Date: 2005-02-05 04:07 Message: Logged In: YES user_id=60314 The problem's easy to fix (using inspect.getmro instead of direct __mro__ access) but hard to reproduce -- what type not recorded in the dispatch table lacks an __mro__? Can't find one easily, and I'd like to have a unit-test reproducing the bug before I commit the fix. Vincenzo, can you explain what the type is happening to be in that value whose deepcopy gives problems? or can anybody else suggest a type as above? ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2005-02-02 22:26 Message: Logged In: YES user_id=29957 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=293338 has a fuller stack trace. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-02-02 14:35 Message: Logged In: YES user_id=6656 Eek, that sounds pretty bad! Can you attach the code/cook up a smaller test case? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1114776&group_id=5470 From noreply at sourceforge.net Thu Apr 14 23:01:54 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 14 23:01:58 2005 Subject: [ python-Bugs-1108992 ] idle freezes when run over ssh Message-ID: Bugs item #1108992, was opened at 2005-01-25 05:44 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1108992&group_id=5470 Category: IDLE Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Mark Poolman (mgpoolman) Assigned to: Nobody/Anonymous (nobody) Summary: idle freezes when run over ssh Initial Comment: Python 2.3.4 (#2, Aug 13 2004, 00:36:58) [GCC 3.3.4 (Debian 1:3.3.4-5ubuntu2)] on linux2 IDLE 1.0.3 When running idle over an ssh link, idle freezes after an unpredictable length of time. Over 3 days the longest it has stayed allive for is ~4hrs, but a few minutes before freezing is the norm. Niether idle nor python are consuming cpu time once frozen. I can find no definete recipie to bring about the freeze, although (I think) there has always been at least one editor window open when it happens. There is no output on stderr, or other diagnostics that I can see. ssh server(ubuntu warty): OpenSSH_3.8.1p1 Debian 1:3.8.1p1-11ubuntu3.1, OpenSSL 0.9.7d 17 Mar 2004 ssh client (RH9): OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f /best/* Mark ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2005-04-14 16:01 Message: Logged In: YES user_id=149084 Any update on this? ---------------------------------------------------------------------- Comment By: SUZUKI Hisao (suzuki_hisao) Date: 2005-03-18 20:46 Message: Logged In: YES user_id=495142 If 1) your IDLE freezes when you close an editor window 2) which has been editing a file whose path contains a non-ASCII character, and 3) you do not call sys.setdefaultencoding() in your sitecustomize.py (so letting the default encoding to be 'ascii'), then my patch 'idlelib.diff' in Python Patch ID 1162825 "EditorWindow's title with non-ASCII chars." may help you. More precisely, IDLE freezes when updating the "Recent Files" menu if an implicit conversion of unicode to ASCII string occurs. The patch fixes it. Sorry if it is irrelevant. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-03-03 19:28 Message: Logged In: YES user_id=149084 There have been recent reports on idle-dev regarding IDLE freezing on Debian Sid. Since ubuntu is Debian derived, I assume there may be a relationship. ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-02-02 13:50 Message: Logged In: YES user_id=993923 >0.8 doesn't have the problem. Are you sure? Can't be certain as haven't used it for extended periods on that box, but I'll look into it. I've used IDLE daily for about 4 years on various RH and Suse, and never seen a problem until now. > What else is the ubuntu box doing? Is the load heavy? Almost nothing, it's there to evaluate ubuntu as a desktop w/s, and my main activity is getting some in-house python s/w ported to it. gdb results to follow. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-02-02 12:21 Message: Logged In: YES user_id=149084 To keep things simple, please start IDLE with the -n switch directly on your ubuntu box for the purposes of future investigations on this issue. Can you attach to the frozen IDLE with gdb and then use 'bt' to get a backtrace and find out what was going at at the time of the freeze? If so, is it repeatable? It's occasionally reported that IDLE freezes. I've never seen it myself, running IDLE for many days on OpenBSD, Debian, RH, and Arch Linux, WindowsXP, W2K, and W98, over a period of many years, so it's hard for me to figure out what's going on. But it's peculiar that 0.8 doesn't have the problem. Are you sure? What else is the ubuntu box doing? Is the load heavy? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-02-01 08:35 Message: Logged In: YES user_id=993923 > So if I follow correctly, IDLE -n freezes on your ubuntu box without using ssh tunneling. That is correct. The problem appears exactly the same when run over ssh though, which, I guess rules out any gnome/metacity/X wierdness. > I suspect a hardware problem I' sceptical about that. There's nothing in dmesg or /var/log to suggest it, and absolutely no other problems with the machine in question. >Are you starting IDLE from the command line Yes, but no messages are shown. > How much memory 255MB, IDLE uses about 3.5% of that on startup, grew to 5.7% at time of last crash ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-01-31 21:01 Message: Logged In: YES user_id=149084 Right, there is no subprocess when you use the -n switch. It causes IDLE to run like IDLE 0.8. So if I follow correctly, IDLE -n freezes on your ubuntu box without using ssh tunneling. I suspect a hardware problem with that system. Are you starting IDLE on the ubuntu box from the command line? If so, are there any messages left in the command line window? How much memory does the ubuntu box have? Does 'top' show increasing memory requirement for the IDLE -n process? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-31 11:46 Message: Logged In: YES user_id=993923 2 - erm forgive me for sounding stupid, but with the -n switch I see no sub-process, only the Idle process. It still freezes though, and killing it just kills it. ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-31 11:27 Message: Logged In: YES user_id=993923 1 - OK It's not ssh it's ubuntu. 2 - I'll try this with ububtu. ...And by frozen, I assume ... All Idle windows are unresponsive to screen and kbd events. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-01-31 01:26 Message: Logged In: YES user_id=149084 Please try two things: 1. Run IDLE directly on the ubuntu machine long enough to assure there is no problem standalone. 2. Run IDLE on the xclient but start it with the -n switch so it runs without the subprocess. When IDLE freezes, walk over to the server and kill the subprocess. It should restart itself. Is the system still frozen? And by frozen, I assume you mean all screen and keyboard events are unresponsive. ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-30 15:06 Message: Logged In: YES user_id=993923 Yes, I've used Idle on that (although only ~ 1 hour) and not seen the problem there. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-01-30 14:43 Message: Logged In: YES user_id=341410 Do you have physical access to the ubuntu warty machine, and if so, does Idle run very well locally on that machine, not X forwarded? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-30 12:02 Message: Logged In: YES user_id=993923 Yes, exactly. I'm pretty sure it's not an X/ssh/network problem, everything else works fine (including Idle 0.8). Mark ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-01-29 15:23 Message: Logged In: YES user_id=341410 It has been a while since I ran Idle, but I am curious as to what you mean by "run over ssh". Do you mean that you have an SSH tunnel to a remote machine, which forwards X-Windows sessions, and when running Idle over this, it locks up? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-26 04:15 Message: Logged In: YES user_id=993923 PS - You don't need an editor window open to get the freeze. M ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1108992&group_id=5470 From noreply at sourceforge.net Fri Apr 15 00:04:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 00:04:25 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 17:14 Message generated for change (Comment added) made by mmm You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- Comment By: Michal Rydlo (mmm) Date: 2005-04-15 00:04 Message: Logged In: YES user_id=65460 foo2.py from #1163244 fails to import. Not being expert in Python internals I hope it is due to this bug. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-11 23:42 Message: Logged In: YES user_id=89016 OK, I'm reopening to bug report. I didn't manage to install pythondoc. cElementTree complains about: No such file or directory: './pyconfig.h'. Can you provide a simple Python file that fails when imported? ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-10 00:47 Message: Logged In: YES user_id=86307 Sorry to comment on a closed report, but perhaps this fix should not be limited only to cases where size is None. Today, I ran into a spurious syntax error when trying to import pythondoc (from http://effbot.org/downloads/pythondoc-2.1b3-20050325.zip). It turned out a \r was ending up in what looked to the parser like the middle of a line, presumably because a \n was dropped. Anyway, I applied the referenced patch to 2.4.1, except I left out the "size is None" condition, since I knew the tokenizer passes in a size param. With that change pythondoc import successfully. (Also, I just ran the test suite and nothing broke.) Since the size parameter is already documented as being passed to StreamReader.read (in codecs.py -- the HTML documentation needs to be updated), and since StreamReader.read says size is an approximate maximum, perhaps it's OK to read one extra byte. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-05 00:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Fri Apr 15 01:16:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 01:16:59 2005 Subject: [ python-Bugs-1114776 ] copy.py bug Message-ID: Bugs item #1114776, was opened at 2005-02-03 01:43 Message generated for change (Settings changed) made by anthonybaxter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1114776&group_id=5470 Category: Python Library Group: Python 2.3 >Status: Closed Resolution: Fixed Priority: 9 Submitted By: Vincenzo Di Somma (disommav) Assigned to: Anthony Baxter (anthonybaxter) Summary: copy.py bug Initial Comment: upgrading from python 2.3.4 to 2.3.5rc1 I got this error on a module that was working correctly File "../Field.py", line 203, in copy properties = deepcopy(cdict) File "/usr/lib/python2.3/copy.py", line 191, in deepcopy y = copier(x, memo) File "/usr/lib/python2.3/copy.py", line 285, in _deepcopy_dict y[deepcopy(key, memo)] = deepcopy(value, memo) File "/usr/lib/python2.3/copy.py", line 200, in deepcopy copier = _getspecial(cls, "__deepcopy__") File "/usr/lib/python2.3/copy.py", line 66, in _getspecial for basecls in cls.__mro__: AttributeError: __mro__ ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-04-15 07:00 Message: Logged In: YES user_id=149084 Should this be closed? ---------------------------------------------------------------------- Comment By: Alex Martelli (aleax) Date: 2005-02-06 18:57 Message: Logged In: YES user_id=60314 Fixed (using inspect.getmro instead of direct __mro__ access) -- added unittest for that thanks to John Lenton's suggestion on c.l.py ---------------------------------------------------------------------- Comment By: Alex Martelli (aleax) Date: 2005-02-05 20:07 Message: Logged In: YES user_id=60314 The problem's easy to fix (using inspect.getmro instead of direct __mro__ access) but hard to reproduce -- what type not recorded in the dispatch table lacks an __mro__? Can't find one easily, and I'd like to have a unit-test reproducing the bug before I commit the fix. Vincenzo, can you explain what the type is happening to be in that value whose deepcopy gives problems? or can anybody else suggest a type as above? ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2005-02-03 14:26 Message: Logged In: YES user_id=29957 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=293338 has a fuller stack trace. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-02-03 06:35 Message: Logged In: YES user_id=6656 Eek, that sounds pretty bad! Can you attach the code/cook up a smaller test case? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1114776&group_id=5470 From noreply at sourceforge.net Fri Apr 15 01:25:29 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 01:25:34 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 06:14 Message generated for change (Comment added) made by glchapman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-14 15:25 Message: Logged In: YES user_id=86307 I think the foo2.py from 1163244 is probably the same bug; at any rate, the reason for it is that a \r is at the beginning of the last line when read in by decoding_fgets. I have simpler test file which shows the bug which I'll email to Walter (you basically just have to get a \r as the last character in the block read by StreamReader, so that atcr will be true). The problem is caused by StreamReader.readline doing: if self.atcr and data.startswith(u"\n"): data = data[1:] since the tokenizer relies on '\n' as the line break character, but it will never see the '\n' removed by the above. FWIW (not much), I think the 2.4 StreamReader.readline actually made more sense than the current code, although a few changes would seem useful (see below). I don't think it is particularly useful to treat the size parameter as a fixed maximum number of bytes to read, since the number of bytes read has no fixed relationship to the number of decoded unicode characters (and also, in the case of the tokenizer, no fixed relationship to the number of bytes of encoded utf8). Also, given the current code, the size parameter is effectively ignored if there is a charbuffer: if you have 5 characters sitting in the charbuffer and use a size of 0x1FF, you only get back the 5 characters, even if they do not end in a linebreak. For the tokenizer, this means an unnecessary PyMem_RESIZE and an extra call to decoding_readline roughly every BUFSIZ bytes in the file (since the tokenizer assumes failure to fetch a complete line means its buffer is too small, whereas in fact it was caused by an incomplete line being stored in the StreamReader's charbuffer). As to changes from 2.4, if the unicode object were to add a findlinebreak method which returns the index of the first character for which Py_UNICODE_ISLINEBREAK is true, readline could use that instead of find("\n"). If it used such a method, readline would also need to explicitly handle a "\r\n" sequence, including a potential read(1) if a '\r' appears at the end of the data (in the case where size is not None). Of course, one problem with that idea is it requires a new method, which may not be allowed until 2.5, and the 2.4.1 behavior definitely needs to be fixed some way. (Interestingly, it looks to me like sre has everything necessary for searching for unicode linebreaks except syntax with which to express the idea in a pattern (maybe I'm missing something, but I can't find a way to get a compiled pattern to emit CATEGORY_UNI_LINEBREAK).) ---------------------------------------------------------------------- Comment By: Michal Rydlo (mmm) Date: 2005-04-14 14:04 Message: Logged In: YES user_id=65460 foo2.py from #1163244 fails to import. Not being expert in Python internals I hope it is due to this bug. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-11 13:42 Message: Logged In: YES user_id=89016 OK, I'm reopening to bug report. I didn't manage to install pythondoc. cElementTree complains about: No such file or directory: './pyconfig.h'. Can you provide a simple Python file that fails when imported? ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-09 14:47 Message: Logged In: YES user_id=86307 Sorry to comment on a closed report, but perhaps this fix should not be limited only to cases where size is None. Today, I ran into a spurious syntax error when trying to import pythondoc (from http://effbot.org/downloads/pythondoc-2.1b3-20050325.zip). It turned out a \r was ending up in what looked to the parser like the middle of a line, presumably because a \n was dropped. Anyway, I applied the referenced patch to 2.4.1, except I left out the "size is None" condition, since I knew the tokenizer passes in a size param. With that change pythondoc import successfully. (Also, I just ran the test suite and nothing broke.) Since the size parameter is already documented as being passed to StreamReader.read (in codecs.py -- the HTML documentation needs to be updated), and since StreamReader.read says size is an approximate maximum, perhaps it's OK to read one extra byte. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-04 14:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Fri Apr 15 01:40:37 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 01:40:48 2005 Subject: [ python-Bugs-1163244 ] Syntax error on large file with MBCS encoding Message-ID: Bugs item #1163244, was opened at 2005-03-14 21:20 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163244&group_id=5470 Category: Parser/Compiler Group: Python 2.4 Status: Open Resolution: Accepted Priority: 7 Submitted By: Tim N. van der Leeuw (tnleeuw) Assigned to: Nobody/Anonymous (nobody) Summary: Syntax error on large file with MBCS encoding Initial Comment: Large files generated by make-py.py from the win32all extensions cannot be compiled by Python2.4.1rc1 - they give a syntax error. This is a regression from 2.3.5 (With Python2.4, the interpreter crashes. That is fixed now.) Removing the mbcs encoding line from the top of the file, compilation succeeds. File should be attached, as zip-file. Probably requires win32all extensions to be installed to be compiled / imported (generated using build 203 of the win32all extensions). ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-15 01:40 Message: Logged In: YES user_id=89016 Importing foo2.py on Linux (with the current CVS HEAD version of Python) gives me a segmentation fault with the following stacktrace: 0x080606cc in instance_repr (inst=0xb7c158bc) at Objects/classobject.c:880 880 classname = inst->in_class->cl_name; (gdb) bt #0 0x080606cc in instance_repr (inst=0xb7c158bc) at Objects/classobject.c:880 #1 0x08082235 in PyObject_Repr (v=0xb7c158bc) at Objects/object.c:308 #2 0x080f3ccd in err_input (err=0xbfffe000) at Python/pythonrun.c:1478 #3 0x080f3956 in PyParser_SimpleParseFileFlags (fp=0x818d6e0, filename=0xbfffe530 "foo2.py", start=257, flags=0) at Python/pythonrun.c:1348 #4 0x080f3982 in PyParser_SimpleParseFile (fp=0x818d6e0, filename=0xbfffe530 "foo2.py", start=257) at Python/pythonrun.c:1355 #5 0x080e6fef in parse_source_module (pathname=0xbfffe530 "foo2.py", fp=0x818d6e0) at Python/import.c:761 #6 0x080e72db in load_source_module (name=0xbfffe9d0 "foo2", pathname=0xbfffe530 "foo2.py", fp=0x818d6e0) at Python/import.c:885 #7 0x080e86b4 in load_module (name=0xbfffe9d0 "foo2", fp=0x818d6e0, buf=0xbfffe530 "foo2.py", type=1, loader=0x0) at Python/import.c:1656 #8 0x080e9d52 in import_submodule (mod=0x8145768, subname=0xbfffe9d0 "foo2", fullname=0xbfffe9d0 "foo2") at Python/import.c:2250 #9 0x080e9511 in load_next (mod=0x8145768, altmod=0x8145768, p_name=0xbfffedf0, buf=0xbfffe9d0 "foo2", p_buflen=0xbfffe9cc) at Python/import.c:2070 #10 0x080e8e5e in import_module_ex (name=0x0, globals=0xb7d62e94, locals=0xb7d62e94, fromlist=0x8145768) at Python/import.c:1905 #11 0x080e914b in PyImport_ImportModuleEx (name=0xb7cd8824 "foo2", globals=0xb7d62e94, locals=0xb7d62e94, fromlist=0x8145768) at Python/import.c:1946 #12 0x080b5c87 in builtin___import__ (self=0x0, args=0xb7d1e634) at Python/bltinmodule.c:45 #13 0x0811d32e in PyCFunction_Call (func=0xb7d523ec, arg=0xb7d1e634, kw=0x0) at Objects/methodobject.c:73 #14 0x0805d188 in PyObject_Call (func=0xb7d523ec, arg=0xb7d1e634, kw=0x0) at Objects/abstract.c:1757 #15 0x080ca79d in PyEval_CallObjectWithKeywords (func=0xb7d523ec, arg=0xb7d1e634, kw=0x0) at Python/ceval.c:3425 #16 0x080c6719 in PyEval_EvalFrame (f=0x816dd7c) at Python/ceval.c:2026 #17 0x080c8fdd in PyEval_EvalCodeEx (co=0xb7cf1ef0, globals=0xb7d62e94, locals=0xb7d62e94, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:2736 #18 0x080bffb0 in PyEval_EvalCode (co=0xb7cf1ef0, globals=0xb7d62e94, locals=0xb7d62e94) at Python/ceval.c:490 #19 0x080f361d in run_node (n=0xb7d122d0, filename=0x8123ba3 "", globals=0xb7d62e94, locals=0xb7d62e94, flags=0xbffff584) at Python/pythonrun.c:1265 #20 0x080f1f58 in PyRun_InteractiveOneFlags (fp=0xb7e94720, filename=0x8123ba3 "", flags=0xbffff584) at Python/pythonrun.c:762 #21 0x080f1c93 in PyRun_InteractiveLoopFlags (fp=0xb7e94720, filename=0x8123ba3 "", flags=0xbffff584) at Python/pythonrun.c:695 #22 0x080f1af6 in PyRun_AnyFileExFlags (fp=0xb7e94720, filename=0x8123ba3 "", closeit=0, flags=0xbffff584) at Python/pythonrun.c:658 #23 0x08055e45 in Py_Main (argc=1, argv=0xbffff634) at Modules/main.c:484 #24 0x08055366 in main (argc=1, argv=0xbffff634) at Modules/python.c:23 The value object in err_input() (in the E_DECODE case) seems to be bogus (it gives me a refcount of -606348325). ---------------------------------------------------------------------- Comment By: Timo Linna (tilinna) Date: 2005-04-09 10:09 Message: Logged In: YES user_id=1074183 Seems that the connection to n*512 blocks is very likely, and it's not just MBCS-related. I managed to reproduce this with a file that contains an ascii-coding declaration, close-to-1024 bytes section, extra crlf and a comment which raises a SyntaxError in Py2.4.1. Could this be linked to the new codec buffering code? See: www.python.org/sf/1178484 ---------------------------------------------------------------------- Comment By: Christos Georgiou (tzot) Date: 2005-03-21 14:34 Message: Logged In: YES user_id=539787 Could be irrelevant but... are the other block sizes close to n*512 (eg 1536 is 3*512) marks? ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2005-03-21 13:11 Message: Logged In: YES user_id=14198 I believe this is a different bug than the recent "long-lines" errors (see below). I can reproduce this with a file that uses neither long lines, nor any pywin32 extensions (2.4 branch, trunk) A Python source file containing: -- start snippet -- # -*- coding: mbcs -*- <1532 characters of code or comments> x = {} -- end snippet -- Will yield a SyntaxError when attempting to import the module. Running the module as a script does not provoke the error. To reproduce, there must be exactly 1532 characters where specified (see the attached file for a demo). Adding or removing even a single character will prevent the error. It is possible to replace characters with any others, including valid code, and still see the error - however, the number of characters must remain the same .cr/lf pairs can also be replaced with any other 2 characters. There are other "block sizes" that will provoke the error, but this is the only one I have nailed. Apart from the "block" of 1532 characters, the coding line and the blank line before the dict assignment also appear critical. Unlike the other characters in the block, this last cr/lf pair can not be replaced with comments. I can't provoke the error with other encodings (note there are no encoded characters in the sample - it is trivial). To reproduce, save the attached file on Windows and execute: > python -c "import foo2" Traceback (most recent call last): File "", line 1, in ? File "foo2.py", line 24 x = {} ^ SyntaxError: invalid syntax Note that Python 2.3 and earlier all work. Also note that "python foo2.py" also works. The code is clearly valid. Haven't tried to repro on Linux (mbcs isn't available there, and I can't get a test case that doesn't use it) Other pointers/notes: pywin32 bug 1085454 is related to long-lines, by all accounts that underlying error has been fixed - I can't verify this as pywin32 no longer generates insanely long lines. I can confirm Python bugs 1101726/1089395 still crashes Python 2.3+. I believe all (including this) are discrete bugs. [foo2.py is my attachment - ya gotta love sourceforge :)] ---------------------------------------------------------------------- Comment By: Christos Georgiou (tzot) Date: 2005-03-20 11:28 Message: Logged In: YES user_id=539787 Useful pointers: in Python-dev, this has been characterised as related to pywin32 bug 1085454. Also related to www.python.org/sf/1101726 and www.python.org/sf/1089395. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163244&group_id=5470 From noreply at sourceforge.net Fri Apr 15 03:53:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 03:53:04 2005 Subject: [ python-Bugs-1183468 ] check for return/yield outside function is wrong Message-ID: Bugs item #1183468, was opened at 2005-04-15 01:53 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183468&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Neil Schemenauer (nascheme) Assigned to: Nobody/Anonymous (nobody) Summary: check for return/yield outside function is wrong Initial Comment: The code currently uses c_nestlevel. That's incorrect as this example code demonstrates: >>> class A: ... return ... Traceback (most recent call last): File "", line 1, in TypeError: Error when calling the metaclass bases PyClass_New: dict must be a dictionary Not sure what the fix is but I wanted to file a bug so it doesn't get forgotten. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183468&group_id=5470 From noreply at sourceforge.net Fri Apr 15 04:23:13 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 04:23:15 2005 Subject: [ python-Bugs-1182603 ] re.escape(s) prints wrong for chr(0) Message-ID: Bugs item #1182603, was opened at 2005-04-13 22:54 Message generated for change (Comment added) made by nascheme You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182603&group_id=5470 Category: Python Library Group: Python 2.4 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nick Jacobson (nickjacobson) Assigned to: Nobody/Anonymous (nobody) Summary: re.escape(s) prints wrong for chr(0) Initial Comment: >>> import re >>> s = chr(0) >>> s '\x00' >>> re.escape(s) '\000' I believe it should print: '\\x00' ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2005-04-15 02:23 Message: Logged In: YES user_id=35752 '\000' is a valid pattern to match the null character. I don't think there is any bug here. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182603&group_id=5470 From noreply at sourceforge.net Fri Apr 15 10:21:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 10:21:52 2005 Subject: [ python-Bugs-1178510 ] configure: refuses setgroups Message-ID: Bugs item #1178510, was opened at 2005-04-07 14:53 Message generated for change (Comment added) made by zosh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: zosh (zosh) Assigned to: Nobody/Anonymous (nobody) Summary: configure: refuses setgroups Initial Comment: On Irix 6.5 with MIPS cc 7.30 configure refuses to recognize the existence of "setgroups". configure:14453: checking for setgroups configure:14477: cc -c -g conftest.c >&5 cc-1020 cc: ERROR File = conftest.c, Line = 159 The identifier "setgroups" is undefined. void* p = setgroups ... 1 error detected in the compilation of "conftest.c". configure:14483: $? = 2 configure: failed program was: | /* confdefs.h. */ ... | #include | #ifdef HAVE_GRP_H | #include | #endif | | int | main () | { | void* p = setgroups | ; | return 0; | } When I compile a short test program that includes only unistd.h with cc, it reports no errors. So I had to manipulate pyconfig.h by hand to force the desired behaviour. Is this a known issue? Please tell me if I have to perform other test on my box to solve the issue. ---------------------------------------------------------------------- >Comment By: zosh (zosh) Date: 2005-04-15 10:21 Message: Logged In: YES user_id=1254741 _SGIAPI is defined in /usr/include/standards.h: #define _SGIAPI ((defined(_SGI_SOURCE) && NO_POSIX && _NO_XOPEN4 && _NO_XOPEN5) || (_ANSIMODE && _NO_POSIX && _NO_XOPEN4 && _NO_XOPEN5)) standards.h contains some comments when e. g. _SGI_SOURCE and other symbols are defined. Refer to the attached file. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-14 20:16 Message: Logged In: YES user_id=21627 Please try to find out whether _SGIAPI is defined anywhere in /usr/include/* or /usr/include/*/*; most likely, some of the -D defines suppress the definition of _SGIAPI. If so, a solution might be to define _SGIAPI in addition to the other defines; see the definition of _GNU_SOURCE, _BSD_VISIBLE etc in configure.in. However, it would be good to find some definition for _SGIAPI - perhaps we are supposed to define _SGI_SOURCE, or _SGI_GIMME_EVERYTHING :-) ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-14 17:30 Message: Logged In: YES user_id=1254741 I must assume that according to the snippet of unistd.h I provided in an earlier comment. Is it possible that setting one -D option clears another one like _SGIAPI ? Tell me please if you would like me perform more tests on my system. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-11 23:21 Message: Logged In: YES user_id=21627 Hmm. You say "because cc should report a non-existent setgroups otherwise"; then you bring an error message '"setgroups" is undefined". Doesn't this imply that _SGIAPI (not _SIGAPI) is indeed not defined, in the case of Python, as well as the case of the various -D options? ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-11 14:29 Message: Logged In: YES user_id=1254741 I believe _SIGAPI is defined beacuse cc should report a non-existent "setgroups" otherwise. Compiling my test program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 resulted in a failure: cc-1020 cc: ERROR File = t.c, Line = 5 The identifier "setgroups" is undefined. void *p = setgroups; ^ ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-09 12:33 Message: Logged In: YES user_id=21627 Why do you believe that _SGIAPI is defined? Please try compiling your program with -D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=600 -D_XOPEN_SOURCE_EXTENDED=1 ---------------------------------------------------------------------- Comment By: zosh (zosh) Date: 2005-04-08 09:51 Message: Logged In: YES user_id=1254741 My test program that compiled correctly with "cc -c -g": #include int main () { void *p = setgroups; return 0; } The definition of setgroups in : #if _SGIAPI ... #if defined(_BSD_COMPAT) extern int setgroups(int, int *); #else /* !_BSD_COMPAT */ extern int setgroups(int, const gid_t *); #endif /* _BSD_COMPAT */ ... #endif /* _SGIAPI */ So it should catch one of these defines. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-08 00:50 Message: Logged In: YES user_id=21627 When you say "I compile a short test program that includes only unistd.h" did you literally mean a program that only includes unistd.h (i.e. has no main function)? If not, what is the short program? In particular, make sure it includes the line void *p = setgroups; Please locate the declaration of setgroups in unistd.h (or whereever it is declared), and report whether this is in a conditional compilation section, and if so, what precisely the condition is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178510&group_id=5470 From noreply at sourceforge.net Fri Apr 15 10:56:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 10:56:34 2005 Subject: [ python-Bugs-1183585 ] try to open /dev/null as directory Message-ID: Bugs item #1183585, was opened at 2005-04-15 10:56 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183585&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Roberto A. Foglietta (robang) Assigned to: Nobody/Anonymous (nobody) Summary: try to open /dev/null as directory Initial Comment: bash-2.05b# strace python 2>&1 | grep open | grep null open("/dev/null", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = -1 ENOTDIR (Not a directory) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183585&group_id=5470 From noreply at sourceforge.net Fri Apr 15 15:37:44 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 15:37:46 2005 Subject: [ python-Bugs-1183742 ] PyDict_Copy() can return non-NULL value on error Message-ID: Bugs item #1183742, was opened at 2005-04-15 14:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183742&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Phil Thompson (philthompson10) Assigned to: Nobody/Anonymous (nobody) Summary: PyDict_Copy() can return non-NULL value on error Initial Comment: PyDict_Copy() will return an invalid pointer rather than NULL if the call to PyDict_Merge() fails. Attached patch fixes it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183742&group_id=5470 From noreply at sourceforge.net Fri Apr 15 16:27:55 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 16:27:56 2005 Subject: [ python-Bugs-1183780 ] Popen4 wait() fails sporadically with threads Message-ID: Bugs item #1183780, was opened at 2005-04-15 14:27 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183780&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Taale Skogan (tskogan) Assigned to: Nobody/Anonymous (nobody) Summary: Popen4 wait() fails sporadically with threads Initial Comment: Calling wait() on a popen2.Popen4 object fails intermittently with the error Traceback (most recent call last): ... File "/usr/local/lib/python2.3/popen2.py", line 90, in wait pid, sts = os.waitpid(self.pid, 0) OSError: [Errno 10] No child processes when using threads. The problem seems to be a race condition when a thread calls wait() on a popen2.Popen4 object. This also apllies to Popen3 objects. The constructor of Popen4. calls _cleanup() which calls poll() which calls the system call waitpid() for all acitve child processes. If another thread calls poll() before the current thread calls wait() on it's child process and the child process has terminated, the child process is no longer waitable and the second call to wait() fails. Code to replicate this behavoir is attached in popen_bug. py. Solution: Popen4 and Popen3 should be threadsafe. Related modules: A seemingly related error occurs with Popen from the new subprocess module. Use the -s option in the popen_bug.py script to test this. Tested on Linux RedHat Enterprise 3 for Python 2.3.3, Python 2.3.5 and Python 2.4.1 and Solaris for Python 2. 4.1. The error did not occur on a RedHat 7.3 machine with Python 2.3.5. See the attached file popen_bug.py for details on the platforms. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183780&group_id=5470 From noreply at sourceforge.net Fri Apr 15 16:40:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 16:41:02 2005 Subject: [ python-Bugs-1183780 ] Popen4 wait() fails sporadically with threads Message-ID: Bugs item #1183780, was opened at 2005-04-15 14:27 Message generated for change (Settings changed) made by tskogan You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183780&group_id=5470 Category: Python Library >Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Taale Skogan (tskogan) Assigned to: Nobody/Anonymous (nobody) Summary: Popen4 wait() fails sporadically with threads Initial Comment: Calling wait() on a popen2.Popen4 object fails intermittently with the error Traceback (most recent call last): ... File "/usr/local/lib/python2.3/popen2.py", line 90, in wait pid, sts = os.waitpid(self.pid, 0) OSError: [Errno 10] No child processes when using threads. The problem seems to be a race condition when a thread calls wait() on a popen2.Popen4 object. This also apllies to Popen3 objects. The constructor of Popen4. calls _cleanup() which calls poll() which calls the system call waitpid() for all acitve child processes. If another thread calls poll() before the current thread calls wait() on it's child process and the child process has terminated, the child process is no longer waitable and the second call to wait() fails. Code to replicate this behavoir is attached in popen_bug. py. Solution: Popen4 and Popen3 should be threadsafe. Related modules: A seemingly related error occurs with Popen from the new subprocess module. Use the -s option in the popen_bug.py script to test this. Tested on Linux RedHat Enterprise 3 for Python 2.3.3, Python 2.3.5 and Python 2.4.1 and Solaris for Python 2. 4.1. The error did not occur on a RedHat 7.3 machine with Python 2.3.5. See the attached file popen_bug.py for details on the platforms. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183780&group_id=5470 From noreply at sourceforge.net Fri Apr 15 18:08:42 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 18:08:45 2005 Subject: [ python-Bugs-1183742 ] PyDict_Copy() can return non-NULL value on error Message-ID: Bugs item #1183742, was opened at 2005-04-15 08:37 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183742&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Phil Thompson (philthompson10) Assigned to: Nobody/Anonymous (nobody) Summary: PyDict_Copy() can return non-NULL value on error Initial Comment: PyDict_Copy() will return an invalid pointer rather than NULL if the call to PyDict_Merge() fails. Attached patch fixes it. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-15 11:08 Message: Logged In: YES user_id=80475 Thanks for the patch. Applied as Objects/dictobject.c 2.164 and 2.162.4.1 . ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183742&group_id=5470 From noreply at sourceforge.net Fri Apr 15 20:16:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 20:16:51 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 17:14 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-15 20:16 Message: Logged In: YES user_id=89016 The current readline() is implemented so that even if the complete line ending (i.e. '\r\n') can't be read within size bytes, at least something that might work as a line end (i.e. '\r') is available at the end of the string, so that the user knowns that the line is complete. The atcr members makes sure that the '\n' that is read next isn't misinterpreted as another line. Unfortunately the import mechanisn doesn't work that way: It demands a '\n' as line terminator and will continue reading if it only finds the '\r'. This means that the '\n' will be skipped and the import mechanisn treats those two lines as one. IMHO the best solution would be the read the extra character even if size is None, as glchapman suggested, so the above situation would really only happen if the last character from the stream is '\r'. I think the tokenizer should be able to survive this. What it didn't survive in 2.4 was that readline() tried to get it's hand on a complete line even if the length of the line was way bigger than the size passed in. If we remove the "size is None" restriction from the current code, then I think we should remove the atcr logic too, because it could only happen under exceptional circumstances and the old handling might be better for those users that don't recognize '\r' as a line end. But in any case the tokenizer should be fixed to be able to handle any line length returned from readline(). I'd really like to get a review by Martin v. L?wis of glchapman's patch #1101726. Of course the simplest solution would be: "If you want a complete line, don't pass a size parameter to readline()". I don't know if it would make sense to change the PEP263 machinery to support this. The other problem is if readline() returns data from the charbuffer. I don't really see how this can be fixed. We might call read() with a size parameter even if there are characters in the charbuffer, but what happens if the user calls readline(size=100000) first and then readline(size=10)? The first readline() call might put a very long string into the charbuffer and then the second call will return a unicode string whose length is in no way correlated to the size parameter. (This happens even even with the current code of course, so the message should be: Don't trust the size parameter, it only tells the underlying stream how many bytes to read (approximately), it's doesn't tell you *anything* about the amount of data returned.). This was different in 2.3.x, because the readline() method of the underlying stream was used, which handled this properly, but only worked if an encoded linefeed was recognizable as a normal 8bit '\n' linefeed by the underlying readline() (i.e. the UTF-16 encodings didn't work). ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-15 01:25 Message: Logged In: YES user_id=86307 I think the foo2.py from 1163244 is probably the same bug; at any rate, the reason for it is that a \r is at the beginning of the last line when read in by decoding_fgets. I have simpler test file which shows the bug which I'll email to Walter (you basically just have to get a \r as the last character in the block read by StreamReader, so that atcr will be true). The problem is caused by StreamReader.readline doing: if self.atcr and data.startswith(u"\n"): data = data[1:] since the tokenizer relies on '\n' as the line break character, but it will never see the '\n' removed by the above. FWIW (not much), I think the 2.4 StreamReader.readline actually made more sense than the current code, although a few changes would seem useful (see below). I don't think it is particularly useful to treat the size parameter as a fixed maximum number of bytes to read, since the number of bytes read has no fixed relationship to the number of decoded unicode characters (and also, in the case of the tokenizer, no fixed relationship to the number of bytes of encoded utf8). Also, given the current code, the size parameter is effectively ignored if there is a charbuffer: if you have 5 characters sitting in the charbuffer and use a size of 0x1FF, you only get back the 5 characters, even if they do not end in a linebreak. For the tokenizer, this means an unnecessary PyMem_RESIZE and an extra call to decoding_readline roughly every BUFSIZ bytes in the file (since the tokenizer assumes failure to fetch a complete line means its buffer is too small, whereas in fact it was caused by an incomplete line being stored in the StreamReader's charbuffer). As to changes from 2.4, if the unicode object were to add a findlinebreak method which returns the index of the first character for which Py_UNICODE_ISLINEBREAK is true, readline could use that instead of find("\n"). If it used such a method, readline would also need to explicitly handle a "\r\n" sequence, including a potential read(1) if a '\r' appears at the end of the data (in the case where size is not None). Of course, one problem with that idea is it requires a new method, which may not be allowed until 2.5, and the 2.4.1 behavior definitely needs to be fixed some way. (Interestingly, it looks to me like sre has everything necessary for searching for unicode linebreaks except syntax with which to express the idea in a pattern (maybe I'm missing something, but I can't find a way to get a compiled pattern to emit CATEGORY_UNI_LINEBREAK).) ---------------------------------------------------------------------- Comment By: Michal Rydlo (mmm) Date: 2005-04-15 00:04 Message: Logged In: YES user_id=65460 foo2.py from #1163244 fails to import. Not being expert in Python internals I hope it is due to this bug. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-11 23:42 Message: Logged In: YES user_id=89016 OK, I'm reopening to bug report. I didn't manage to install pythondoc. cElementTree complains about: No such file or directory: './pyconfig.h'. Can you provide a simple Python file that fails when imported? ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-10 00:47 Message: Logged In: YES user_id=86307 Sorry to comment on a closed report, but perhaps this fix should not be limited only to cases where size is None. Today, I ran into a spurious syntax error when trying to import pythondoc (from http://effbot.org/downloads/pythondoc-2.1b3-20050325.zip). It turned out a \r was ending up in what looked to the parser like the middle of a line, presumably because a \n was dropped. Anyway, I applied the referenced patch to 2.4.1, except I left out the "size is None" condition, since I knew the tokenizer passes in a size param. With that change pythondoc import successfully. (Also, I just ran the test suite and nothing broke.) Since the size parameter is already documented as being passed to StreamReader.read (in codecs.py -- the HTML documentation needs to be updated), and since StreamReader.read says size is an approximate maximum, perhaps it's OK to read one extra byte. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-05 00:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Fri Apr 15 21:16:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 21:16:58 2005 Subject: [ python-Bugs-1183959 ] return val in __init__ doesn't raise TypeError in new-style Message-ID: Bugs item #1183959, was opened at 2005-04-15 22:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183959&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Adal Chiriliuc (adalx) Assigned to: Nobody/Anonymous (nobody) Summary: return val in __init__ doesn't raise TypeError in new-style Initial Comment: returning a value (other than None) from __init__ is supposed to raise TypeError. This only works for old-style classes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183959&group_id=5470 From noreply at sourceforge.net Fri Apr 15 21:20:07 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 21:20:11 2005 Subject: [ python-Bugs-1183959 ] return val in __init__ doesn't raise TypeError in new-style Message-ID: Bugs item #1183959, was opened at 2005-04-15 22:16 Message generated for change (Comment added) made by adalx You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183959&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Adal Chiriliuc (adalx) Assigned to: Nobody/Anonymous (nobody) Summary: return val in __init__ doesn't raise TypeError in new-style Initial Comment: returning a value (other than None) from __init__ is supposed to raise TypeError. This only works for old-style classes. ---------------------------------------------------------------------- >Comment By: Adal Chiriliuc (adalx) Date: 2005-04-15 22:20 Message: Logged In: YES user_id=1067739 # I've failed to attach this file so I've added a comment. This should be modified to report eventual failure and added to the test library. class LegalConstructor1(object): def __init__(self): return class LegalConstructor2(object): def __init__(self): return None class IlegalConstructor(object): def __init__(self): return "x" class LegalConstructorOld1: def __init__(self): return class LegalConstructorOld2: def __init__(self): return None class IlegalConstructorOld: def __init__(self): return "x" a = LegalConstructor1() print a b = LegalConstructor2() print b x = LegalConstructorOld1() print a y = LegalConstructorOld2() print b # !!! This should raise TypeError c = IlegalConstructor() print c z = IlegalConstructorOld() print z ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183959&group_id=5470 From noreply at sourceforge.net Fri Apr 15 21:39:32 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 15 21:39:35 2005 Subject: [ python-Bugs-1183972 ] dest parameter in optparse Message-ID: Bugs item #1183972, was opened at 2005-04-15 14:39 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183972&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: ahmado (ahmado) Assigned to: Nobody/Anonymous (nobody) Summary: dest parameter in optparse Initial Comment: When using the "callback" action, optparse requires that you STILL specify a "dest" parameter, though it is not required or used by the option parser. In fact, if you do not include the "dest" parameter, your options will still work properly, but optparse will generate an exception when calling the "--help" option. Not specifying a "dest" parameter will result in the following Exception when using the default "--help" or "- h" option: File "c:\Python23\lib\optparse.py", line 229, in format_option_strings metavar = option.metavar or option.dest.upper() AttributeError: 'NoneType' object has no attribute 'upper' by simply adding a bogus bug, the --help / -h option works properly even though the dest isn't actually used. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183972&group_id=5470 From noreply at sourceforge.net Sat Apr 16 03:55:06 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 16 03:55:08 2005 Subject: [ python-Bugs-1184112 ] Missing trailing newline with comment raises SyntaxError Message-ID: Bugs item #1184112, was opened at 2005-04-15 18:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184112&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Eric Huss (ehuss) Assigned to: Nobody/Anonymous (nobody) Summary: Missing trailing newline with comment raises SyntaxError Initial Comment: The following illustrates a problem with the parser handling the lack of trailing newlines: >>> parser.suite('def foo():\n\tpass\n\n# comment') Traceback (most recent call last): File "", line 1, in ? File "", line 4 # comment ^ SyntaxError: invalid syntax >>> parser.suite('def foo():\n\tpass\n\n# comment\n') This is similar to bug 501622, however, this only seems to happen when you have an indented block, followed by a comment line that has no trailing newline. I traced through tokenizer.c and whittled down the issue into tok_get(). In the statement where it is processing the comment character and looking at the tabforms, in the first case this will end up with 'c' equal to EOF whereas in the second case "c" will eqaul '\n'. When it equals EOF, it is unable to do the cleanup necessary to emit the DEDENT token (it immediately bails out with ENDMARKER which causes parsetok() to barf because the indentation level is still 1 inside tok_state). Attached is a patch of a little hack I made that seems to fix the problem. Although it seems to be a safe thing to do, it is definitely a hack. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184112&group_id=5470 From noreply at sourceforge.net Sat Apr 16 03:57:33 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 16 03:57:36 2005 Subject: [ python-Bugs-1184112 ] Missing trailing newline with comment raises SyntaxError Message-ID: Bugs item #1184112, was opened at 2005-04-15 18:55 Message generated for change (Comment added) made by ehuss You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184112&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Eric Huss (ehuss) Assigned to: Nobody/Anonymous (nobody) Summary: Missing trailing newline with comment raises SyntaxError Initial Comment: The following illustrates a problem with the parser handling the lack of trailing newlines: >>> parser.suite('def foo():\n\tpass\n\n# comment') Traceback (most recent call last): File "", line 1, in ? File "", line 4 # comment ^ SyntaxError: invalid syntax >>> parser.suite('def foo():\n\tpass\n\n# comment\n') This is similar to bug 501622, however, this only seems to happen when you have an indented block, followed by a comment line that has no trailing newline. I traced through tokenizer.c and whittled down the issue into tok_get(). In the statement where it is processing the comment character and looking at the tabforms, in the first case this will end up with 'c' equal to EOF whereas in the second case "c" will eqaul '\n'. When it equals EOF, it is unable to do the cleanup necessary to emit the DEDENT token (it immediately bails out with ENDMARKER which causes parsetok() to barf because the indentation level is still 1 inside tok_state). Attached is a patch of a little hack I made that seems to fix the problem. Although it seems to be a safe thing to do, it is definitely a hack. ---------------------------------------------------------------------- >Comment By: Eric Huss (ehuss) Date: 2005-04-15 18:57 Message: Logged In: YES user_id=393416 Well, wonderful sourceforge is barfing with the error "ArtifactFile: Could not open file for writing" when trying to upload my patch, so I'll just post it in the comment here. Very sorry. :( --- tokenizer.c 3 Feb 2004 22:53:59 -0000 1.2 +++ tokenizer.c 16 Apr 2005 01:45:05 -0000 @@ -1139,6 +1139,9 @@ } while (c != EOF && c != '\n') c = tok_nextc(tok); + if (c == EOF) { + c = '\n'; + } } /* Check for EOF and errors now */ ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184112&group_id=5470 From noreply at sourceforge.net Sat Apr 16 09:54:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 16 09:54:50 2005 Subject: [ python-Bugs-1183585 ] try to open /dev/null as directory Message-ID: Bugs item #1183585, was opened at 2005-04-15 08:56 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183585&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Roberto A. Foglietta (robang) Assigned to: Nobody/Anonymous (nobody) Summary: try to open /dev/null as directory Initial Comment: bash-2.05b# strace python 2>&1 | grep open | grep null open("/dev/null", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = -1 ENOTDIR (Not a directory) ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2005-04-16 07:54 Message: Logged In: YES user_id=849994 I don't quite understand what you're trying to tell us here. Are you just calling Python this way and observing the system calls? Then, what system do you use? What version? What kernel, etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183585&group_id=5470 From noreply at sourceforge.net Sat Apr 16 20:52:05 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 16 20:52:08 2005 Subject: [ python-Bugs-1182614 ] dir() does not include _ Message-ID: Bugs item #1182614, was opened at 2005-04-13 18:21 Message generated for change (Settings changed) made by nickjacobson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182614&group_id=5470 Category: None Group: Python 2.4 >Status: Closed Resolution: None Priority: 5 Submitted By: Nick Jacobson (nickjacobson) Assigned to: Nobody/Anonymous (nobody) Summary: dir() does not include _ Initial Comment: At the interpreter prompt, dir() does not include the variable _ >>> dir() ['__builtins__', '__doc__', '__name__'] >>> x = 5 >>> dir() ['__builtins__', '__doc__', '__name__', 'x'] >>> _ ['__builtins__', '__doc__', '__name__', 'x'] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1182614&group_id=5470 From noreply at sourceforge.net Sat Apr 16 21:21:07 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 16 21:21:11 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 00:56 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-16 12:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-30 22:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Sat Apr 16 21:36:25 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 16 21:36:27 2005 Subject: [ python-Bugs-1184380 ] example broken in section 1.12 of Extending & Embedding Message-ID: Bugs item #1184380, was opened at 2005-04-16 14:36 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184380&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: bamoore (bamoore) Assigned to: Nobody/Anonymous (nobody) Summary: example broken in section 1.12 of Extending & Embedding Initial Comment: In section 1.12 of Extending and Embedding the Python Interpreter Release 2.4, the example (spammodule) does not compile on linux with distutils. I get the following: ~/sandbox/python/c_api_test$ python2.4 spam-setup.py build running build running build_ext building 'spam' extension creating build creating build/temp.linux-i686-2.4 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.4 -c spammodule.c -o build/temp.linux-i686-2.4/spammodule.o spammodule.c:7: error: conflicting types for `PySpam_System' spammodule.h:21: error: previous declaration of `PySpam_System' error: command 'gcc' failed with exit status 1 I think the solution is to change line 12 of the spammodule header file listing from: #define PySpam_System_PROTO (char *command) to: #define PySpam_System_PROTO (const char *command) as that seems to compile for me. (though not tested from another C extension). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184380&group_id=5470 From noreply at sourceforge.net Sat Apr 16 22:27:17 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 16 22:27:24 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 17:56 Message generated for change (Comment added) made by jbelmonte You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- Comment By: John Belmonte (jbelmonte) Date: 2005-04-17 05:27 Message: Logged In: YES user_id=282299 This problem also occurs on Linux. Note however that I've traced the source of the symptom to a different bug fix, namely [1010677] thread Module Breaks PyGILState_Ensure() (http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470). Specifically, the change from v2.56 to v2.56.8.1 of threadmodule.c. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-17 04:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-31 15:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Sun Apr 17 00:05:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 17 00:05:24 2005 Subject: [ python-Bugs-1183468 ] check for return/yield outside function is wrong Message-ID: Bugs item #1183468, was opened at 2005-04-14 20:53 Message generated for change (Comment added) made by logistix You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183468&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Neil Schemenauer (nascheme) Assigned to: Nobody/Anonymous (nobody) Summary: check for return/yield outside function is wrong Initial Comment: The code currently uses c_nestlevel. That's incorrect as this example code demonstrates: >>> class A: ... return ... Traceback (most recent call last): File "", line 1, in TypeError: Error when calling the metaclass bases PyClass_New: dict must be a dictionary Not sure what the fix is but I wanted to file a bug so it doesn't get forgotten. ---------------------------------------------------------------------- Comment By: logistix (logistix) Date: 2005-04-16 17:05 Message: Logged In: YES user_id=699438 Patch 1184418 is (sortof) posted. SF is giving me grief when I try to do it as a real attachment though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183468&group_id=5470 From noreply at sourceforge.net Sun Apr 17 01:16:50 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 17 01:16:53 2005 Subject: [ python-Bugs-1184449 ] Read-only property attributes raise wrong exception Message-ID: Bugs item #1184449, was opened at 2005-04-16 19:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Barry A. Warsaw (bwarsaw) Assigned to: Raymond Hettinger (rhettinger) Summary: Read-only property attributes raise wrong exception Initial Comment: In http://mail.python.org/pipermail/python-dev/2005-April/052681.html I describe how attributes defined as properties raise inconsistent errors depending on whether the property is defined in C or Python. By default in C, you'll get a TypeError while in Python you'll get an AttributeError. I think both should raise AttributeErrors. Here is a patch against Python 2.4 which should be ported to 2.5 (and possibly 2.3 if we care). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 From noreply at sourceforge.net Sun Apr 17 01:23:43 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 17 01:23:45 2005 Subject: [ python-Bugs-1184449 ] Read-only property attributes raise wrong exception Message-ID: Bugs item #1184449, was opened at 2005-04-16 19:16 Message generated for change (Comment added) made by bwarsaw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Barry A. Warsaw (bwarsaw) Assigned to: Raymond Hettinger (rhettinger) Summary: Read-only property attributes raise wrong exception Initial Comment: In http://mail.python.org/pipermail/python-dev/2005-April/052681.html I describe how attributes defined as properties raise inconsistent errors depending on whether the property is defined in C or Python. By default in C, you'll get a TypeError while in Python you'll get an AttributeError. I think both should raise AttributeErrors. Here is a patch against Python 2.4 which should be ported to 2.5 (and possibly 2.3 if we care). ---------------------------------------------------------------------- >Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-16 19:23 Message: Logged In: YES user_id=12800 Well, I was going to attach a patch, but it seems SF is borked and won't let me upload anything (yes the stupid checkbox was checked). The patch is simple though: change PyExc_TypeError to PyExc_AttributeError on lines 147 and 202 in Objects/descrobject.c (rev 2.38) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 From noreply at sourceforge.net Sun Apr 17 12:42:38 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 17 12:42:43 2005 Subject: [ python-Bugs-1173637 ] quit should quit Message-ID: Bugs item #1173637, was opened at 2005-03-30 21:37 Message generated for change (Comment added) made by pernici You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 Category: None Group: None Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Matt Chaput (mchaput) Assigned to: Nobody/Anonymous (nobody) Summary: quit should quit Initial Comment: When the user types "quit" in the interpreter, instead of quitting the program gives him or her a lecture on the "proper" way to quit. This is very obnoxious. Since the interpreter obviously understands the "quit" command, it should just quit, dammit. ---------------------------------------------------------------------- Comment By: Pernici Mario (pernici) Date: 2005-04-17 10:42 Message: Logged In: YES user_id=756712 To allow the builtin quit() only when using Python interactively, one can put in a start-up quitfile.py def quit(): import sys sys.exit(0) and set export PYTHONSTARTUP=quitfile.py One could add to this file also exit = quit since the builtin exit behaves like quit. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-02 12:24 Message: Logged In: YES user_id=4771 Raymond, an argument against quit() actually quitting the interpreter is that it suddenly makes quit() a quasi-official piece of the Python API, and we're bound to see user programs start to write quit() instead of sys.exit(). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-01 21:47 Message: Logged In: YES user_id=21627 But how is this better than the current >>> quit 'Use Ctrl-D (i.e. EOF) to exit.' I'd rather learn that I have to type Ctrl-D instead of typing quit() I think the original report is misguided: The interpreter does not "obviously understand" the quit command. Instead, it really does not understand it all. The name quit is bound to a string, and the interpreter displays the string. It does not understand at all what the string says. So I'm rejecting the report as "won't fix". ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 18:13 Message: Logged In: YES user_id=80475 class quit: def __repr__(self): return "Type quit() to exit the interpreter" def __call__(self): sys.exit() Requiring the parentheses is not a big deal as long as there is an advisory where they are omitted. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2005-04-01 11:12 Message: Logged In: YES user_id=4771 This discussion keeps coming up time and again. sjoerd's trick has been tried a few times already, until people implementing it realized that trying to display the builtins dict (directly or indirectly) would quit the interpreter! 'quit' cannot just be a synonym for 'sys.exit' either, because typing 'quit' would just print '', which is worse than the current situation in term of expliciteness. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2005-04-01 11:04 Message: Logged In: YES user_id=43607 Something like this instead of the current quit should do the trick: class Quit: def __repr__(self): import sys sys.exit(0) quit = Quit() del Quit The problem with the Raymond's suggestion is that you need to type parentheses. But of course, this does get a little magical... ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-01 06:14 Message: Logged In: YES user_id=80475 'quit' and 'exit' currently show-up in a dir() of__builtins__. If the OP's suggestion is accepted, it should probably be implemented just like a builtin: def quit(): sys.exit() That approach is not at all magical and still allows shadowing (quit=2, etc.) What we have now is an annoying wart. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-01 04:35 Message: Logged In: YES user_id=971153 I am not sure adding quit to interpreter is such a good idea: Right now quit is treated as an ordinary (non-keyword) identifier... (with a bit of magic: if quit is not defined then lecture the user :-))... E.g now you can do this: >>> quit=2 >>> quit 2 Would you want to disallow this usage when quit becomes a "magic word"? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-03-31 09:49 Message: Logged In: YES user_id=6656 I'm not so sure typing quit should quit -- that doesn't seem like Python to me (how would you implement it?) Having quit be the same object as sys.exit seems sensible. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-03-30 22:11 Message: Logged In: YES user_id=80475 I concur! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173637&group_id=5470 From noreply at sourceforge.net Sun Apr 17 16:07:10 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 17 16:07:13 2005 Subject: [ python-Bugs-1184380 ] example broken in section 1.12 of Extending & Embedding Message-ID: Bugs item #1184380, was opened at 2005-04-16 14:36 Message generated for change (Settings changed) made by bamoore You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184380&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None >Priority: 1 Submitted By: bamoore (bamoore) Assigned to: Nobody/Anonymous (nobody) Summary: example broken in section 1.12 of Extending & Embedding Initial Comment: In section 1.12 of Extending and Embedding the Python Interpreter Release 2.4, the example (spammodule) does not compile on linux with distutils. I get the following: ~/sandbox/python/c_api_test$ python2.4 spam-setup.py build running build running build_ext building 'spam' extension creating build creating build/temp.linux-i686-2.4 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.4 -c spammodule.c -o build/temp.linux-i686-2.4/spammodule.o spammodule.c:7: error: conflicting types for `PySpam_System' spammodule.h:21: error: previous declaration of `PySpam_System' error: command 'gcc' failed with exit status 1 I think the solution is to change line 12 of the spammodule header file listing from: #define PySpam_System_PROTO (char *command) to: #define PySpam_System_PROTO (const char *command) as that seems to compile for me. (though not tested from another C extension). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184380&group_id=5470 From noreply at sourceforge.net Sun Apr 17 18:05:38 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Apr 17 18:05:41 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 19:05 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Mon Apr 18 01:08:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 01:08:49 2005 Subject: [ python-Bugs-1184449 ] Read-only property attributes raise wrong exception Message-ID: Bugs item #1184449, was opened at 2005-04-16 18:16 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 Category: Python Interpreter Core >Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Barry A. Warsaw (bwarsaw) Assigned to: Raymond Hettinger (rhettinger) Summary: Read-only property attributes raise wrong exception Initial Comment: In http://mail.python.org/pipermail/python-dev/2005-April/052681.html I describe how attributes defined as properties raise inconsistent errors depending on whether the property is defined in C or Python. By default in C, you'll get a TypeError while in Python you'll get an AttributeError. I think both should raise AttributeErrors. Here is a patch against Python 2.4 which should be ported to 2.5 (and possibly 2.3 if we care). ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-16 18:23 Message: Logged In: YES user_id=12800 Well, I was going to attach a patch, but it seems SF is borked and won't let me upload anything (yes the stupid checkbox was checked). The patch is simple though: change PyExc_TypeError to PyExc_AttributeError on lines 147 and 202 in Objects/descrobject.c (rev 2.38) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 From noreply at sourceforge.net Mon Apr 18 01:39:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 01:39:50 2005 Subject: [ python-Bugs-1112856 ] patch 1079734 broke cgi.FieldStorage w/ multipart post req. Message-ID: Bugs item #1112856, was opened at 2005-01-30 16:58 Message generated for change (Comment added) made by joshhoyt You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112856&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Nobody/Anonymous (nobody) Summary: patch 1079734 broke cgi.FieldStorage w/ multipart post req. Initial Comment: Patch 1079734 "Make cgi.py use email instead of rfc822 or mimetools" seems to have broken the cgi.FieldStorage in cases where the request is a multipart post (for instance, when a file upload form field is used). See the attached test program. With cgi.py revision <1.83 (python 2.4 for instance) I get the expected results; 374 FieldStorage(None, None, [FieldStorage('param1', None, 'Value of param1'), FieldStorage('param2', None, 'Value of param2'), FieldStorage('file', '', ''), FieldStorage(None, None, '')]) but with cgi.py rev 1.83 (current) I get this: 374 FieldStorage(None, None, [FieldStorage('param1', None, '')]) Another thing that I observed (which isn't reproduced by this test program) is that cgi.FieldStorage.__init__ never completes when the fp is a socket-file (and the request is again a multipart post). It worked fine with the old cgi.py. ---------------------------------------------------------------------- Comment By: Josh Hoyt (joshhoyt) Date: 2005-04-17 16:39 Message: Logged In: YES user_id=693077 I've been playing with using the email parser to do the whole job of parsing the data, and I think it's too big of a change. It'd be very hard to ensure API compatibility and the same behaviour. I think that in the long run, it's the right thing to do, but I think that the API should change when that change is made. My recommendation for the time being is to revert the original patch that I made. ---------------------------------------------------------------------- Comment By: Irmen de Jong (irmen) Date: 2005-02-08 13:40 Message: Logged In: YES user_id=129426 I've added a test that shows the 'freezing' problem I talked about. Start the server.py, it will listen on port 9000. Open the post.html in your web browser, enter some form data, and submit the form. It will POST to the server.py and if you started that one with cvs-python (2.5a0) it will freeze on the marked line. If you start server.py with Python 2.4, it will work fine. ---------------------------------------------------------------------- Comment By: Irmen de Jong (irmen) Date: 2005-02-06 14:58 Message: Logged In: YES user_id=129426 Yes, I'll try to make a test case for that within the next few days. ---------------------------------------------------------------------- Comment By: Josh Hoyt (joshhoyt) Date: 2005-02-04 15:02 Message: Logged In: YES user_id=693077 Irmen, can you try to create a test case where the cgi.FieldStorage never completes, so I can make sure that any fix I come up with resolves it? I will try to put together an implementation where the email parser parses the whole multipart message. ---------------------------------------------------------------------- Comment By: Irmen de Jong (irmen) Date: 2005-02-04 14:06 Message: Logged In: YES user_id=129426 Johannes: while your patch makes my cgibug.py test case run fine, it has 2 problems: 1- it runs much slower than the python2.4 code (probably because of the reading back thing Josh is talking about); 2- it still doesn't fix the second problem that I observed: cgi.FieldStorage never completes when fp is a socket. I don't have a separate test case for this yet, sorry. So Josh: perhaps your idea doesn't have these 2 problems? ---------------------------------------------------------------------- Comment By: Josh Hoyt (joshhoyt) Date: 2005-02-04 07:45 Message: Logged In: YES user_id=693077 Johannes, your patch looks fine to me. It would be nice if we didn't have to keep reading back each part from the parsed message, though. I had an idea for another approach. Use email to parse the MIME message fully, then convert it to FieldStorage fields. Parsing could go something like: == CODE == from email.FeedParser import FeedParser parser = FeedParser() # Create bogus content-type header... parser.feed('Content-type: %s ; boundary=%s \r\n\r\n' % (self.type, self.innerboundary)) parser.feed(self.fp.read()) message = parser.close() # Then take parsed message and convert to FieldStorage fields == END CODE == This lets the email parser handle all of the complexities of MIME, but it does mean that we have to accurately re-create all of the necessary headers. I can cook up a full patch if anyone thinks this would fly. ---------------------------------------------------------------------- Comment By: Johannes Gijsbers (jlgijsbers) Date: 2005-02-04 02:15 Message: Logged In: YES user_id=469548 Here's a patch. We're interested in two things in the patched loop: * the rest of the multipart/form-data, including the headers for the current part, for consumption by HeaderParser (this is the tail variable) * the rest of the multipart/form-data without the headers for the current part, for consumption by FieldStorage (this is the message.get_payload() call) Josh, Irmen, do you see any problems with this patch? (BTW, this fix should be ported to the parse_multipart function as well, when I check it in (and I should make cgibug.py into a test)) ---------------------------------------------------------------------- Comment By: Josh Hoyt (joshhoyt) Date: 2005-02-03 23:21 Message: Logged In: YES user_id=693077 The email.HeaderParser module that is now used for parsing headers consumes the rest of the body of the message when it is used. If there was some way of telling the parser to stop reading when it got to the end of the headers, I think this problem would be fixed. Unfortunately, there is no external interface to the email module's parser that lets us do this. I hope we can come up with a reasonable solution that does not involve reverting to using deprecated modules. Thanks for the test case. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1112856&group_id=5470 From noreply at sourceforge.net Mon Apr 18 10:39:13 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 10:39:21 2005 Subject: [ python-Bugs-808596 ] apparently leaking snippet Message-ID: Bugs item #808596, was opened at 2003-09-18 15:48 Message generated for change (Settings changed) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=808596&group_id=5470 Category: None Group: Python 2.3 Status: Open Resolution: Later Priority: 2 Submitted By: Michael Hudson (mwh) Assigned to: Nobody/Anonymous (nobody) >Summary: apparently leaking snippet Initial Comment: Raymond Hettinger found that the following: fd, fname = tempfile.mkstemp() execfile(fname) leaks a reference. TrackRefs sez it's a string. Haven't dug any further. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-08-09 14:46 Message: Logged In: YES user_id=6656 Reopening, as it turned out interning filenames did do some good. Lowering priority as it isn't a real leak. It's still kinda annoying, though. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-08-03 11:25 Message: Logged In: YES user_id=6656 Done, in Python/compile.c revision 2.312. test_pkgimport turned out to be something else (sys.path_importer_cache), so I feel no need to try and be clever with dummy's refcounting today. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-08-02 16:11 Message: Logged In: YES user_id=80475 Not interning the filename seems cleanest. Like you, I do not expect that interning provided any real benefit here. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-08-02 13:56 Message: Logged In: YES user_id=6656 I've finally worked out what's going on here. It's because compiling the file interns the name of the file being compiled. Now that interned strings are mortal, this isn't a big deal, but it still results in a string being added to and removed from the interned strings dictionary with net result (almost all of the time) of incrementing the refcount on 'dummy' by one. Once in every thousand or so iterations this results in the interned string dictionary being resized and throwing all the references to dummy away, so this isn't a 'real' leak. OTOH, I'd still like to do something about this (as is, test_pkg appears to leak 10 references per run because of this). The simplest thing would be to just plain not intern the filename during compilation (I find it hard to believe that it's of any real benefit). The more invasive solution would be to not do refcounting on dummy (assuming that's even possible) which is a scarier change, but would stop similar problems with other tests (I have a feeling that test_pkgimport appears to be leaking references because of a different incarnation of the same problem). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=808596&group_id=5470 From noreply at sourceforge.net Mon Apr 18 10:50:44 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 10:50:49 2005 Subject: [ python-Bugs-1176893 ] Readline segfault Message-ID: Bugs item #1176893, was opened at 2005-04-05 09:50 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 Category: Python Library Group: Python 2.5 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Walter D?rwald (doerwalter) >Assigned to: Michael Hudson (mwh) Summary: Readline segfault Initial Comment: The latest change to the readline module has broken tab completion: ./python Python 2.5a0 (#1, Apr 5 2005, 01:14:33) [GCC 3.3.5 (Gentoo Linux 3.3.5-r1, ssp-3.3.2-3, pie-8.7.7.1)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import readline, rlcompleter [25913 refs] >>> readline.parse_and_bind("tab: complete") [25913 refs] >>> Segmentation fault [Press tab after the parse_and_bind() call] ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-18 09:50 Message: Logged In: YES user_id=6656 Fixed in Python/pystate.c revision 2.40 Misc/NEWS revision 1.1285 If Mark is actually listening and has any comments, I'd still be interested to hear them. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-07 16:15 Message: Logged In: YES user_id=31435 Sorry, can't make time for this. Reassigned to Mark in case he can. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-05 11:27 Message: Logged In: YES user_id=6656 Or maybe this one line patch is the answer (it certainly fixes this case). Tim, can you spare a minute to think about this? The patch simply adds a check to PyEval_ReleaseThread that doesn't call PyThread_release_lock if the GIL hasn't been allocated. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-05 11:18 Message: Logged In: YES user_id=6656 I'm going to go out on a limb and suggest this is a bug in the PyGilState_ functions. The problem burrows down to calling PyThread_release_lock(interpreter_lock) when interpreter_lock is NULL, i.e. PyEval_InitThreads hasn't been called. (if you start a thread before running the crashing code, it doesn't crash, because the GIL has been allocated). Not sure what the solution is, or who to bug (time to read cvs log, I guess). A silly workaround is to put PyEval_InitThreads in initreadline. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1176893&group_id=5470 From noreply at sourceforge.net Mon Apr 18 14:11:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 14:11:27 2005 Subject: [ python-Bugs-1185121 ] itertools.imerge: merge sequences Message-ID: Bugs item #1185121, was opened at 2005-04-18 12:11 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185121&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Jurjen N.E. Bos (jneb) Assigned to: Nobody/Anonymous (nobody) Summary: itertools.imerge: merge sequences Initial Comment: (For the itertools library, so Python 2.2 and up) This is a suggested addition to itertools, proposed name imerge. usage: imerge(seq0, seq1, ..., [key=]) result: imerge assumes the sequences are all in sorted order, and produces a iterator that returns pairs of the form (value, index), where value is a value of one of the sequences, and index is the index number of the given sequence. The output the imerge is in sorted order (taking into account the key function), so that identical values in the sequences will be produced from left to right. The code is surprisingly short, making use of the builtin heap module. (You may disagree with my style of argument handling; feel free to optimize it.) def imerge(*iterlist, **key): """Merge a sequence of sorted iterables. Returns pairs [value, index] where each value comes from iterlist[index], and the pairs are sorted if each of the iterators is sorted. Hint use groupby(imerge(...), operator.itemgetter(0)) to get the items one by one. """ if key.keys() not in ([], ["key"]): raise TypeError, "Excess keyword arguments for imerge" key = key.get("key", lambda x:x) from heapq import heapreplace, heappop #initialize the heap containing (inited, value, index, currentItem, iterator) #this automatically makes sure all iterators are initialized, then run, and finally emptied heap = [(False, None, index, None, iter(iterator)) for index, iterator in enumerate(iterlist)] while heap: inited, item, index, value, iterator = heap[0] if inited: yield value, index try: item = iterator.next() except StopIteration: heappop(heap) else: heapreplace(heap, (True, key(item), index, item, iterator)) If you find this little routine worth its size, please put it into itertools. - Jurjen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185121&group_id=5470 From noreply at sourceforge.net Mon Apr 18 14:18:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 14:18:42 2005 Subject: [ python-Bugs-1185124 ] pydoc doesn't find all module doc strings Message-ID: Bugs item #1185124, was opened at 2005-04-18 12:18 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Kent Johnson (kjohnson) Assigned to: Nobody/Anonymous (nobody) Summary: pydoc doesn't find all module doc strings Initial Comment: pydoc.synopsis() attempts to find a module's doc string by parsing the module text. But the parser only recognizes strings created with """ and r""". Any other docstring is ignored. I've attached a patch against Python 2.4.1 that fixes pydoc to recognize ''' and r''' strings but really it should recognize any allowable string format. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 From noreply at sourceforge.net Mon Apr 18 18:51:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 18:51:17 2005 Subject: [ python-Feature Requests-1185121 ] itertools.imerge: merge sequences Message-ID: Feature Requests item #1185121, was opened at 2005-04-18 07:11 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 >Category: Python Library >Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jurjen N.E. Bos (jneb) >Assigned to: Raymond Hettinger (rhettinger) Summary: itertools.imerge: merge sequences Initial Comment: (For the itertools library, so Python 2.2 and up) This is a suggested addition to itertools, proposed name imerge. usage: imerge(seq0, seq1, ..., [key=]) result: imerge assumes the sequences are all in sorted order, and produces a iterator that returns pairs of the form (value, index), where value is a value of one of the sequences, and index is the index number of the given sequence. The output the imerge is in sorted order (taking into account the key function), so that identical values in the sequences will be produced from left to right. The code is surprisingly short, making use of the builtin heap module. (You may disagree with my style of argument handling; feel free to optimize it.) def imerge(*iterlist, **key): """Merge a sequence of sorted iterables. Returns pairs [value, index] where each value comes from iterlist[index], and the pairs are sorted if each of the iterators is sorted. Hint use groupby(imerge(...), operator.itemgetter(0)) to get the items one by one. """ if key.keys() not in ([], ["key"]): raise TypeError, "Excess keyword arguments for imerge" key = key.get("key", lambda x:x) from heapq import heapreplace, heappop #initialize the heap containing (inited, value, index, currentItem, iterator) #this automatically makes sure all iterators are initialized, then run, and finally emptied heap = [(False, None, index, None, iter(iterator)) for index, iterator in enumerate(iterlist)] while heap: inited, item, index, value, iterator = heap[0] if inited: yield value, index try: item = iterator.next() except StopIteration: heappop(heap) else: heapreplace(heap, (True, key(item), index, item, iterator)) If you find this little routine worth its size, please put it into itertools. - Jurjen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 From noreply at sourceforge.net Mon Apr 18 19:05:33 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 19:05:36 2005 Subject: [ python-Bugs-1184449 ] Read-only property attributes raise wrong exception Message-ID: Bugs item #1184449, was opened at 2005-04-16 18:16 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 Category: Python Interpreter Core Group: Python 2.5 Status: Open >Resolution: Accepted Priority: 5 Submitted By: Barry A. Warsaw (bwarsaw) >Assigned to: Barry A. Warsaw (bwarsaw) Summary: Read-only property attributes raise wrong exception Initial Comment: In http://mail.python.org/pipermail/python-dev/2005-April/052681.html I describe how attributes defined as properties raise inconsistent errors depending on whether the property is defined in C or Python. By default in C, you'll get a TypeError while in Python you'll get an AttributeError. I think both should raise AttributeErrors. Here is a patch against Python 2.4 which should be ported to 2.5 (and possibly 2.3 if we care). ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-16 18:23 Message: Logged In: YES user_id=12800 Well, I was going to attach a patch, but it seems SF is borked and won't let me upload anything (yes the stupid checkbox was checked). The patch is simple though: change PyExc_TypeError to PyExc_AttributeError on lines 147 and 202 in Objects/descrobject.c (rev 2.38) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 From noreply at sourceforge.net Mon Apr 18 19:11:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 19:11:23 2005 Subject: [ python-Bugs-1183959 ] return val in __init__ doesn't raise TypeError in new-style Message-ID: Bugs item #1183959, was opened at 2005-04-15 14:16 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183959&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Adal Chiriliuc (adalx) Assigned to: Nobody/Anonymous (nobody) Summary: return val in __init__ doesn't raise TypeError in new-style Initial Comment: returning a value (other than None) from __init__ is supposed to raise TypeError. This only works for old-style classes. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 12:11 Message: Logged In: YES user_id=80475 This has been fixed for 2.4.1 where a RuntimeWarning is raised and in 2.5.0a where a TypeError is raised. ---------------------------------------------------------------------- Comment By: Adal Chiriliuc (adalx) Date: 2005-04-15 14:20 Message: Logged In: YES user_id=1067739 # I've failed to attach this file so I've added a comment. This should be modified to report eventual failure and added to the test library. class LegalConstructor1(object): def __init__(self): return class LegalConstructor2(object): def __init__(self): return None class IlegalConstructor(object): def __init__(self): return "x" class LegalConstructorOld1: def __init__(self): return class LegalConstructorOld2: def __init__(self): return None class IlegalConstructorOld: def __init__(self): return "x" a = LegalConstructor1() print a b = LegalConstructor2() print b x = LegalConstructorOld1() print a y = LegalConstructorOld2() print b # !!! This should raise TypeError c = IlegalConstructor() print c z = IlegalConstructorOld() print z ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183959&group_id=5470 From noreply at sourceforge.net Mon Apr 18 19:12:17 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 19:12:20 2005 Subject: [ python-Bugs-1185124 ] pydoc doesn't find all module doc strings Message-ID: Bugs item #1185124, was opened at 2005-04-18 07:18 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Kent Johnson (kjohnson) >Assigned to: Ka-Ping Yee (ping) Summary: pydoc doesn't find all module doc strings Initial Comment: pydoc.synopsis() attempts to find a module's doc string by parsing the module text. But the parser only recognizes strings created with """ and r""". Any other docstring is ignored. I've attached a patch against Python 2.4.1 that fixes pydoc to recognize ''' and r''' strings but really it should recognize any allowable string format. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 From noreply at sourceforge.net Mon Apr 18 20:23:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 20:23:39 2005 Subject: [ python-Bugs-1185124 ] pydoc doesn't find all module doc strings Message-ID: Bugs item #1185124, was opened at 2005-04-18 05:18 Message generated for change (Comment added) made by ping You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Kent Johnson (kjohnson) Assigned to: Ka-Ping Yee (ping) Summary: pydoc doesn't find all module doc strings Initial Comment: pydoc.synopsis() attempts to find a module's doc string by parsing the module text. But the parser only recognizes strings created with """ and r""". Any other docstring is ignored. I've attached a patch against Python 2.4.1 that fixes pydoc to recognize ''' and r''' strings but really it should recognize any allowable string format. ---------------------------------------------------------------------- >Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 11:23 Message: Logged In: YES user_id=45338 PEP 257 recommends: "For consistency, always use """triple double quotes""" around docstrings." I think that's why this was originally written to only look for triple double-quotes. Are there a large number of modules written using triple-single quotes for the module docstring? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 From noreply at sourceforge.net Mon Apr 18 20:29:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 20:29:42 2005 Subject: [ python-Bugs-1178148 ] cgitb.py report wrong line number Message-ID: Bugs item #1178148, was opened at 2005-04-06 16:04 Message generated for change (Comment added) made by ping You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178148&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open >Resolution: Works For Me Priority: 5 Submitted By: Barry Alan Scott (barry-scott) >Assigned to: Ka-Ping Yee (ping) Summary: cgitb.py report wrong line number Initial Comment: Given code like try: raise 'bug' except ValueError: pass # cgitb.py think 'bug' is here cgitb.py will report that the exception 'bug' is at the pass line. This is a time waster until you figure that this problem exists. ---------------------------------------------------------------------- >Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 11:29 Message: Logged In: YES user_id=45338 Can't reproduce this problem. When i test the provided case, cgitb reports the error on the "raise" line as it should. See the attached file for my test case; cgitb highlights line 6 ("raise 'spam'"). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178148&group_id=5470 From noreply at sourceforge.net Mon Apr 18 21:26:50 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 21:26:53 2005 Subject: [ python-Feature Requests-1185383 ] Make bisect.* functions accept an optional compare function Message-ID: Feature Requests item #1185383, was opened at 2005-04-18 21:26 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185383&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Marcin Ciura (mciura) Assigned to: Nobody/Anonymous (nobody) Summary: Make bisect.* functions accept an optional compare function Initial Comment: The usability of bisect.bisect_right, bisect.bisect_left, bisect.insort_right and bisect.insort_left would increase if they accepted an optional less_than function to compare the keys. Here is my use case: I have a sorted list of reversed words and a parallel list of flags associated with these words (taken from ispell). The list of reversed words is the one searched; the flags are the result of a search. Issue #1: Now I cannot use simply a list of tuples (rev_word,flags) and a custom less_than function that compares only the first elements of two tuples. Issue #2: When a word is not found in the list, I'd like to make an educated guess as to its flags (this makes more sense in non-English languages, where e.g. infinitives have a special ending), using bisect_left and bisect_right: from bisect import * less_than = lambda x,y: x[:3] Bugs item #1071094, was opened at 2004-11-22 16:15 Message generated for change (Comment added) made by bmaret You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1071094&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Marc-Antoine Parent (maparent) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: some latex reject the pdfinfo macro while generating html Initial Comment: I was building the documentation on OSX. I use the latest LaTeX 2004 from Wierda's teTeX. There is a pdfinfo command in Doc/ texinputs/manual.cls and Doc/texinputs/howto.cls which chokes my version of latex. edited log follows: TEXINPUTS=/.../Python-2.4c1/Doc/commontex: python /.../Python -2.4c1/Doc/tools/mkhowto --html --about html/stdabout.dat -- iconserver ../icons --favicon ../icons/pyfav.png --address "See About this document... for information on suggesting changes." --up-link ../index.html --up- title "Python Documentation Index" --global-module-index "../ modindex.html" --dvips-safe --dir html/api api/api.tex +++ TEXINPUTS=/.../Python-2.4c1/ Doc/api:/.../Python-2.4c1/Doc/ commontex:/.../Python-2.4c1/Doc/ paper-letter:/.../Python-2.4c1/Doc/ texinputs: +++ latex api *** Session transcript and error messages are in /.../Python -2.4c1/Doc/html/api/api.how. *** Exited with status 1. The relevant lines from the transcript are: ------------------------------------------------------------------------ +++ latex api This is pdfeTeX, Version 3.141592-1.20a-2.2 (Web2C 7.5.3) output format initialized to DVI entering extended mode (/.../Python-2.4c1/Doc/api/api.tex LaTeX2e <2003/12/01> Babel and hyphenation patterns for american, french, german, ngerman, nohyphenation, loaded. (/.../Python-2.4c1/Doc/texinputs/manual.cls Document Class: manual 1998/03/03 Document class (Python manual) (/.../Python-2.4c1/Doc/texinputs/pypaper.sty (/usr/local/teTeX/share/texmf.tetex/tex/latex/psnfss/times.sty) Using Times instead of Computer Modern. ) (/usr/local/teTeX/share/texmf.tetex/tex/latex/fancybox/ fancybox.sty Style option: `fancybox' v1.3 <2000/09/19> (tvz) ) (/usr/local/teTeX/share/texmf.tetex/tex/latex/base/report.cls Document Class: report 2004/02/16 v1.4f Standard LaTeX document class (/usr/local/teTeX/share/texmf.tetex/tex/latex/base/size10.clo)) (/.../Python-2.4c1/Doc/texinputs/fancyhdr.sty) Using fancier footers than usual. (/.../Python-2.4c1/Doc/texinputs/fncychap.sty) Using fancy chapter headings. (/.../Python-2.4c1/Doc/texinputs/python.sty (/usr/local/teTeX/share/texmf.tetex/tex/latex/tools/longtable.sty) (/usr/local/teTeX/share/texmf.tetex/tex/latex/tools/verbatim.sty) (/usr/local/teTeX/share/texmf.tetex/tex/latex/base/alltt.sty))) (/.../Python-2.4c1/Doc/commontex/boilerplate.tex (/.../Python-2.4c1/Doc/commontex/patchlevel.tex)) Writing index file api.idx No file api.aux. (/usr/local/teTeX/share/texmf.tetex/tex/latex/psnfss/ot1ptm.fd) pdfTeX error (ext1): \pdfinfo used while \pdfoutput is not set. { \def \{, } \pdfinfo { /Author (\@author ) /Title (\@title ) } } l.12 \maketitle No pages of output. Transcript written on api.log. *** Session transcript and error messages are in /.../Python -2.4c1/Doc/html/api/api.how. *** Exited with status 1. make: *** [html/api/api.html] Error 1 ---------------------------------------------------------------------- Comment By: S?bastien Maret (bmaret) Date: 2005-04-18 19:53 Message: Logged In: YES user_id=842097 I get the exact same error. I am running MacOSX with teTeX 3.0 installed with Fink. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1071094&group_id=5470 From noreply at sourceforge.net Mon Apr 18 22:04:06 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 22:04:09 2005 Subject: [ python-Bugs-1185124 ] pydoc doesn't find all module doc strings Message-ID: Bugs item #1185124, was opened at 2005-04-18 12:18 Message generated for change (Comment added) made by kjohnson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Kent Johnson (kjohnson) Assigned to: Ka-Ping Yee (ping) Summary: pydoc doesn't find all module doc strings Initial Comment: pydoc.synopsis() attempts to find a module's doc string by parsing the module text. But the parser only recognizes strings created with """ and r""". Any other docstring is ignored. I've attached a patch against Python 2.4.1 that fixes pydoc to recognize ''' and r''' strings but really it should recognize any allowable string format. ---------------------------------------------------------------------- >Comment By: Kent Johnson (kjohnson) Date: 2005-04-18 20:04 Message: Logged In: YES user_id=49695 I don't know if there are a large number of modules with triple-single-quoted docstrings. Pydoc will search any module in site-packages at least, so you have to consider third-party modules. At best pydoc is inconsistent - the web browser display uses the __doc__attribute but search and apropos use synopsis(). It's a pretty simple change to recognize any triple-quoted string, it seems like a good idea to me... I have attached a revised patch that uses a regex match so it works with e.g. uR""" and other variations of triple-quoting. FWIW this bug report was motivated by this thread on comp.lang.python: http://groups-beta.google.com/group/comp.lang.python/browse_frm/thread/e5cfccb7c9a168d7/1c1702e71e1939b0?q=triple&rnum=1#1c1702e71e1939b0 ---------------------------------------------------------------------- Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 18:23 Message: Logged In: YES user_id=45338 PEP 257 recommends: "For consistency, always use """triple double quotes""" around docstrings." I think that's why this was originally written to only look for triple double-quotes. Are there a large number of modules written using triple-single quotes for the module docstring? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 From noreply at sourceforge.net Mon Apr 18 22:09:09 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 22:09:12 2005 Subject: [ python-Feature Requests-1185383 ] Make bisect.* functions accept an optional compare function Message-ID: Feature Requests item #1185383, was opened at 2005-04-18 14:26 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185383&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Marcin Ciura (mciura) Assigned to: Nobody/Anonymous (nobody) Summary: Make bisect.* functions accept an optional compare function Initial Comment: The usability of bisect.bisect_right, bisect.bisect_left, bisect.insort_right and bisect.insort_left would increase if they accepted an optional less_than function to compare the keys. Here is my use case: I have a sorted list of reversed words and a parallel list of flags associated with these words (taken from ispell). The list of reversed words is the one searched; the flags are the result of a search. Issue #1: Now I cannot use simply a list of tuples (rev_word,flags) and a custom less_than function that compares only the first elements of two tuples. Issue #2: When a word is not found in the list, I'd like to make an educated guess as to its flags (this makes more sense in non-English languages, where e.g. infinitives have a special ending), using bisect_left and bisect_right: from bisect import * less_than = lambda x,y: x[:3]Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 15:09 Message: Logged In: YES user_id=80475 A few thoughts: * If bisect provided an optional lt argument, it would still not be thread-safe. The code for the lt function can call arbitrary Python code that runs through the eval-loop and is hence subject to a thread-switch. * An advantage of the class wrapper approach is that the prefix function gets computed just once per word. This is not a big deal for the specific case of [:3], but it could be a significant benefit for other functions that are expensive to compute (because repeated calls to bisect will access the lt function more than once). * My own approach to the particular use case would be to map prefixes to a list of revwords or (revword, flag) pairs: d = dict(abb=['abbc'], abc=['abcaa', 'abcdd', 'abcss'], abd=['abdf']) This gives O(1) lookup time and limits the calls to the prefix function to once per word. Will leave this request open as it may yet be a good idea. My concern is that it will clutter the code and the API for only a small benefit. Also, I'm looking at a more general purpose solution that would make this feature request moot. This idea is to create a fast comparison decorator class used like this: dwordlist = map(ComparisonDecorator(lambda x: x[:3]), wordlist) lp = bisect_left(dwordlist, given_rev_word) rp = bisect_right(dwordlist, given_rev_word) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185383&group_id=5470 From noreply at sourceforge.net Mon Apr 18 22:28:17 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 22:28:19 2005 Subject: [ python-Bugs-1185124 ] pydoc doesn't find all module doc strings Message-ID: Bugs item #1185124, was opened at 2005-04-18 05:18 Message generated for change (Comment added) made by ping You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Kent Johnson (kjohnson) Assigned to: Ka-Ping Yee (ping) Summary: pydoc doesn't find all module doc strings Initial Comment: pydoc.synopsis() attempts to find a module's doc string by parsing the module text. But the parser only recognizes strings created with """ and r""". Any other docstring is ignored. I've attached a patch against Python 2.4.1 that fixes pydoc to recognize ''' and r''' strings but really it should recognize any allowable string format. ---------------------------------------------------------------------- >Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 13:28 Message: Logged In: YES user_id=45338 I think you're right that if it works for the module summary (using __doc__) then it should work with synopsis(). However, the patch you've added doesn't address the problem properly; instead of handling """ correctly and ignoring ''', it handles both kinds of docstrings incorrectly because it will accept ''' as a match for """ or """ as a match for '''. I'll look at fixing this soon, but feel free to keep prodding me until it gets fixed. ---------------------------------------------------------------------- Comment By: Kent Johnson (kjohnson) Date: 2005-04-18 13:04 Message: Logged In: YES user_id=49695 I don't know if there are a large number of modules with triple-single-quoted docstrings. Pydoc will search any module in site-packages at least, so you have to consider third-party modules. At best pydoc is inconsistent - the web browser display uses the __doc__attribute but search and apropos use synopsis(). It's a pretty simple change to recognize any triple-quoted string, it seems like a good idea to me... I have attached a revised patch that uses a regex match so it works with e.g. uR""" and other variations of triple-quoting. FWIW this bug report was motivated by this thread on comp.lang.python: http://groups-beta.google.com/group/comp.lang.python/browse_frm/thread/e5cfccb7c9a168d7/1c1702e71e1939b0?q=triple&rnum=1#1c1702e71e1939b0 ---------------------------------------------------------------------- Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 11:23 Message: Logged In: YES user_id=45338 PEP 257 recommends: "For consistency, always use """triple double quotes""" around docstrings." I think that's why this was originally written to only look for triple double-quotes. Are there a large number of modules written using triple-single quotes for the module docstring? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 From noreply at sourceforge.net Mon Apr 18 23:11:01 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 23:11:02 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 11:05 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 16:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Mon Apr 18 23:15:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 18 23:16:02 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 19:05 Message generated for change (Comment added) made by poromenos You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Poromenos (poromenos) Date: 2005-04-19 00:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 00:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Tue Apr 19 00:20:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 00:20:17 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 11:05 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:20 Message: Logged In: YES user_id=80475 I'm -1 on complicating str.replace()'s API for functionality that substantially duplicates that offered by str.translate(): >>> "Hello world".translate(string.maketrans('ed', 'ae')) 'Hallo worle' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-18 16:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 16:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Tue Apr 19 00:23:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 00:23:17 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 19:05 Message generated for change (Comment added) made by poromenos You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Poromenos (poromenos) Date: 2005-04-19 01:23 Message: Logged In: YES user_id=267121 Ah, I did not know that... The docs are a bit complicated on . translate, but since it can do that, yes, it would be unwise to implement my suggestion. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 01:20 Message: Logged In: YES user_id=80475 I'm -1 on complicating str.replace()'s API for functionality that substantially duplicates that offered by str.translate(): >>> "Hello world".translate(string.maketrans('ed', 'ae')) 'Hallo worle' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-19 00:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 00:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Tue Apr 19 00:43:30 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 00:43:33 2005 Subject: [ python-Feature Requests-1185121 ] itertools.imerge: merge sequences Message-ID: Feature Requests item #1185121, was opened at 2005-04-18 07:11 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jurjen N.E. Bos (jneb) Assigned to: Raymond Hettinger (rhettinger) Summary: itertools.imerge: merge sequences Initial Comment: (For the itertools library, so Python 2.2 and up) This is a suggested addition to itertools, proposed name imerge. usage: imerge(seq0, seq1, ..., [key=]) result: imerge assumes the sequences are all in sorted order, and produces a iterator that returns pairs of the form (value, index), where value is a value of one of the sequences, and index is the index number of the given sequence. The output the imerge is in sorted order (taking into account the key function), so that identical values in the sequences will be produced from left to right. The code is surprisingly short, making use of the builtin heap module. (You may disagree with my style of argument handling; feel free to optimize it.) def imerge(*iterlist, **key): """Merge a sequence of sorted iterables. Returns pairs [value, index] where each value comes from iterlist[index], and the pairs are sorted if each of the iterators is sorted. Hint use groupby(imerge(...), operator.itemgetter(0)) to get the items one by one. """ if key.keys() not in ([], ["key"]): raise TypeError, "Excess keyword arguments for imerge" key = key.get("key", lambda x:x) from heapq import heapreplace, heappop #initialize the heap containing (inited, value, index, currentItem, iterator) #this automatically makes sure all iterators are initialized, then run, and finally emptied heap = [(False, None, index, None, iter(iterator)) for index, iterator in enumerate(iterlist)] while heap: inited, item, index, value, iterator = heap[0] if inited: yield value, index try: item = iterator.next() except StopIteration: heappop(heap) else: heapreplace(heap, (True, key(item), index, item, iterator)) If you find this little routine worth its size, please put it into itertools. - Jurjen ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:43 Message: Logged In: YES user_id=80475 I had previously looked at an imerge() utility and found that it had only a single application (isomorphic to lazy mergesorting) and that the use cases were dominated by the in-memory alternative: sorted(chain(*iterlist)). Short of writing an external mergesort, what applications did you have in mind? What situations have you encountered where you have multiple sources of sorted data being generated on the fly (as opposed to already being in-memory), have needed one element at a time sequential access to a combined sort of that data, needed that combined sort only once, and could not afford to have the dataset in-memory? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 From noreply at sourceforge.net Tue Apr 19 00:48:08 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 00:48:13 2005 Subject: [ python-Feature Requests-1175686 ] add "reload" function Message-ID: Feature Requests item #1175686, was opened at 2005-04-03 03:48 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1175686&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) >Assigned to: Kurt B. Kaiser (kbk) Summary: add "reload" function Initial Comment: The file menu in Editor windows could benefit from a "reload" function that re-reads the file from disc and loads it into the editor, replacing what's currently in the editor. That's useful for those of us who like to edit with Emacs in another window while running the code under Idle. To get really fancy, it might be cool to optionally display a diff between the in-editor version and the on-disk version allowing merging changes, but that's maybe over the top. Other editors usually do have commands like this, e.g.M-x Revert-file in Emacs or ":e!" in vi. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:48 Message: Logged In: YES user_id=80475 If added, it should definitely be called "revert" instead of "reload". The latter already has a specific meaning in Python that relates to imports. The former seems to be standard terminology across a number of editing tools from emacs to photoshop. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1175686&group_id=5470 From noreply at sourceforge.net Tue Apr 19 01:16:09 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 01:16:12 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 11:05 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-18 17:23 Message: Logged In: YES user_id=267121 Ah, I did not know that... The docs are a bit complicated on . translate, but since it can do that, yes, it would be unwise to implement my suggestion. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:20 Message: Logged In: YES user_id=80475 I'm -1 on complicating str.replace()'s API for functionality that substantially duplicates that offered by str.translate(): >>> "Hello world".translate(string.maketrans('ed', 'ae')) 'Hallo worle' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-18 16:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 16:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Tue Apr 19 04:55:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 04:55:30 2005 Subject: [ python-Feature Requests-1175686 ] add "reload" function Message-ID: Feature Requests item #1175686, was opened at 2005-04-03 03:48 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1175686&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: paul rubin (phr) Assigned to: Kurt B. Kaiser (kbk) Summary: add "reload" function Initial Comment: The file menu in Editor windows could benefit from a "reload" function that re-reads the file from disc and loads it into the editor, replacing what's currently in the editor. That's useful for those of us who like to edit with Emacs in another window while running the code under Idle. To get really fancy, it might be cool to optionally display a diff between the in-editor version and the on-disk version allowing merging changes, but that's maybe over the top. Other editors usually do have commands like this, e.g.M-x Revert-file in Emacs or ":e!" in vi. ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2005-04-18 21:55 Message: Logged In: YES user_id=149084 Why do you use Emacs instead of IDLE? (I use emacs myself sometimes, but I'm curious what your reasons are these days.) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:48 Message: Logged In: YES user_id=80475 If added, it should definitely be called "revert" instead of "reload". The latter already has a specific meaning in Python that relates to imports. The former seems to be standard terminology across a number of editing tools from emacs to photoshop. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1175686&group_id=5470 From noreply at sourceforge.net Tue Apr 19 07:26:58 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 07:27:04 2005 Subject: [ python-Bugs-1166714 ] [ast branch] fatal error when compiling test_bool.py Message-ID: Bugs item #1166714, was opened at 2005-03-19 16:14 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: John Ehresman (jpe) Assigned to: Brett Cannon (bcannon) Summary: [ast branch] fatal error when compiling test_bool.py Initial Comment: When using the ast-branch compiler, python exits with a fatal error when compiling test_bool.py or the symplified test case that I'll attach. The immediate problem is that when the compiler is in make_closure after compiling the lambda, self is found to be a LOCAL reference type and make_closure assumes it's a FREE reference if it's not a CELL reference in the enclosing scope. I don't know if self is supposed to be a LOCAL reference or if make_closure should handle LOCAL references. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-18 22:26 Message: Logged In: YES user_id=357491 John, on line 45 of the patch, which becomes line 580 in Python/symtable.c, you call PyDict_Update() with three arguments. Is 'local' supposed to be an argument, or is 'free' the erroneous one? If I change it so that 'local' is removed then the tests die on test_unicode (although a ton of other tests fail). ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 17:22 Message: Logged In: YES user_id=22785 Forgot to add the patch ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 17:21 Message: Logged In: YES user_id=22785 The attached patch seems to fix this by creating a dictionary for each scope's free vars and only propagating up the ones that aren't satisfied by a binding. With this and the other patches, the test suite runs w/o segfaulting up through test_unicode. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 13:37 Message: Logged In: YES user_id=22785 The issue seems to be that in symtable_analyze, a free dictionary is created for the entire module and if a name is found to be free in any function, it is not converted from a LOCAL to a CELL is any context seen afterwards. I think the free variables need to be recorded on a function-by-function basis. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-03-22 12:34 Message: Logged In: YES user_id=357491 An short example of the failure:: def outer(a): def first(): a def last(a): pass This leads to a fatal Python error. The issue is having a being a local or parameter (can make 'a' be 'a = 1' and not a parameter and still have it fail), the first closure reference it, and then the second one use it as a parameter. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 From noreply at sourceforge.net Tue Apr 19 10:19:09 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 10:19:14 2005 Subject: [ python-Feature Requests-1185121 ] itertools.imerge: merge sequences Message-ID: Feature Requests item #1185121, was opened at 2005-04-18 12:11 Message generated for change (Comment added) made by jneb You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jurjen N.E. Bos (jneb) Assigned to: Raymond Hettinger (rhettinger) Summary: itertools.imerge: merge sequences Initial Comment: (For the itertools library, so Python 2.2 and up) This is a suggested addition to itertools, proposed name imerge. usage: imerge(seq0, seq1, ..., [key=]) result: imerge assumes the sequences are all in sorted order, and produces a iterator that returns pairs of the form (value, index), where value is a value of one of the sequences, and index is the index number of the given sequence. The output the imerge is in sorted order (taking into account the key function), so that identical values in the sequences will be produced from left to right. The code is surprisingly short, making use of the builtin heap module. (You may disagree with my style of argument handling; feel free to optimize it.) def imerge(*iterlist, **key): """Merge a sequence of sorted iterables. Returns pairs [value, index] where each value comes from iterlist[index], and the pairs are sorted if each of the iterators is sorted. Hint use groupby(imerge(...), operator.itemgetter(0)) to get the items one by one. """ if key.keys() not in ([], ["key"]): raise TypeError, "Excess keyword arguments for imerge" key = key.get("key", lambda x:x) from heapq import heapreplace, heappop #initialize the heap containing (inited, value, index, currentItem, iterator) #this automatically makes sure all iterators are initialized, then run, and finally emptied heap = [(False, None, index, None, iter(iterator)) for index, iterator in enumerate(iterlist)] while heap: inited, item, index, value, iterator = heap[0] if inited: yield value, index try: item = iterator.next() except StopIteration: heappop(heap) else: heapreplace(heap, (True, key(item), index, item, iterator)) If you find this little routine worth its size, please put it into itertools. - Jurjen ---------------------------------------------------------------------- >Comment By: Jurjen N.E. Bos (jneb) Date: 2005-04-19 08:19 Message: Logged In: YES user_id=446428 Well, I was optimizing a piece of code with reasonbly long sorted lists (in memory, I agree) that were modified in all kinds of ways. I did not want the nlogn behaviour of sort, so I started writing a merge routine. I found out that the boundary cases of a merge implementation are a mess, until I disccovered the heap trick. Then I decided to clean it up and and put it up for a library routine. The fact that it uses iterators is obnly to make it more general, not specifically for the "lazy" properties. - Jurjen ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 22:43 Message: Logged In: YES user_id=80475 I had previously looked at an imerge() utility and found that it had only a single application (isomorphic to lazy mergesorting) and that the use cases were dominated by the in-memory alternative: sorted(chain(*iterlist)). Short of writing an external mergesort, what applications did you have in mind? What situations have you encountered where you have multiple sources of sorted data being generated on the fly (as opposed to already being in-memory), have needed one element at a time sequential access to a combined sort of that data, needed that combined sort only once, and could not afford to have the dataset in-memory? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 From noreply at sourceforge.net Tue Apr 19 13:05:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 13:05:28 2005 Subject: [ python-Bugs-1108992 ] idle freezes when run over ssh Message-ID: Bugs item #1108992, was opened at 2005-01-25 10:44 Message generated for change (Comment added) made by mgpoolman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1108992&group_id=5470 Category: IDLE Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Mark Poolman (mgpoolman) Assigned to: Nobody/Anonymous (nobody) Summary: idle freezes when run over ssh Initial Comment: Python 2.3.4 (#2, Aug 13 2004, 00:36:58) [GCC 3.3.4 (Debian 1:3.3.4-5ubuntu2)] on linux2 IDLE 1.0.3 When running idle over an ssh link, idle freezes after an unpredictable length of time. Over 3 days the longest it has stayed allive for is ~4hrs, but a few minutes before freezing is the norm. Niether idle nor python are consuming cpu time once frozen. I can find no definete recipie to bring about the freeze, although (I think) there has always been at least one editor window open when it happens. There is no output on stderr, or other diagnostics that I can see. ssh server(ubuntu warty): OpenSSH_3.8.1p1 Debian 1:3.8.1p1-11ubuntu3.1, OpenSSL 0.9.7d 17 Mar 2004 ssh client (RH9): OpenSSH_3.5p1, SSH protocols 1.5/2.0, OpenSSL 0x0090701f /best/* Mark ---------------------------------------------------------------------- >Comment By: Mark Poolman (mgpoolman) Date: 2005-04-19 11:05 Message: Logged In: YES user_id=993923 Haven't been using the machine in question for a while. I can't reproduce the problem on an opteron with equivalent set up. (The machine I reported this on is a celeron). I've atatched a stack trace from gdb, having interrupted with ctrl-C at the gdb prompt when idle froze. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-04-14 21:01 Message: Logged In: YES user_id=149084 Any update on this? ---------------------------------------------------------------------- Comment By: SUZUKI Hisao (suzuki_hisao) Date: 2005-03-19 01:46 Message: Logged In: YES user_id=495142 If 1) your IDLE freezes when you close an editor window 2) which has been editing a file whose path contains a non-ASCII character, and 3) you do not call sys.setdefaultencoding() in your sitecustomize.py (so letting the default encoding to be 'ascii'), then my patch 'idlelib.diff' in Python Patch ID 1162825 "EditorWindow's title with non-ASCII chars." may help you. More precisely, IDLE freezes when updating the "Recent Files" menu if an implicit conversion of unicode to ASCII string occurs. The patch fixes it. Sorry if it is irrelevant. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-03-04 00:28 Message: Logged In: YES user_id=149084 There have been recent reports on idle-dev regarding IDLE freezing on Debian Sid. Since ubuntu is Debian derived, I assume there may be a relationship. ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-02-02 18:50 Message: Logged In: YES user_id=993923 >0.8 doesn't have the problem. Are you sure? Can't be certain as haven't used it for extended periods on that box, but I'll look into it. I've used IDLE daily for about 4 years on various RH and Suse, and never seen a problem until now. > What else is the ubuntu box doing? Is the load heavy? Almost nothing, it's there to evaluate ubuntu as a desktop w/s, and my main activity is getting some in-house python s/w ported to it. gdb results to follow. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-02-02 17:21 Message: Logged In: YES user_id=149084 To keep things simple, please start IDLE with the -n switch directly on your ubuntu box for the purposes of future investigations on this issue. Can you attach to the frozen IDLE with gdb and then use 'bt' to get a backtrace and find out what was going at at the time of the freeze? If so, is it repeatable? It's occasionally reported that IDLE freezes. I've never seen it myself, running IDLE for many days on OpenBSD, Debian, RH, and Arch Linux, WindowsXP, W2K, and W98, over a period of many years, so it's hard for me to figure out what's going on. But it's peculiar that 0.8 doesn't have the problem. Are you sure? What else is the ubuntu box doing? Is the load heavy? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-02-01 13:35 Message: Logged In: YES user_id=993923 > So if I follow correctly, IDLE -n freezes on your ubuntu box without using ssh tunneling. That is correct. The problem appears exactly the same when run over ssh though, which, I guess rules out any gnome/metacity/X wierdness. > I suspect a hardware problem I' sceptical about that. There's nothing in dmesg or /var/log to suggest it, and absolutely no other problems with the machine in question. >Are you starting IDLE from the command line Yes, but no messages are shown. > How much memory 255MB, IDLE uses about 3.5% of that on startup, grew to 5.7% at time of last crash ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-02-01 02:01 Message: Logged In: YES user_id=149084 Right, there is no subprocess when you use the -n switch. It causes IDLE to run like IDLE 0.8. So if I follow correctly, IDLE -n freezes on your ubuntu box without using ssh tunneling. I suspect a hardware problem with that system. Are you starting IDLE on the ubuntu box from the command line? If so, are there any messages left in the command line window? How much memory does the ubuntu box have? Does 'top' show increasing memory requirement for the IDLE -n process? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-31 16:46 Message: Logged In: YES user_id=993923 2 - erm forgive me for sounding stupid, but with the -n switch I see no sub-process, only the Idle process. It still freezes though, and killing it just kills it. ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-31 16:27 Message: Logged In: YES user_id=993923 1 - OK It's not ssh it's ubuntu. 2 - I'll try this with ububtu. ...And by frozen, I assume ... All Idle windows are unresponsive to screen and kbd events. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-01-31 06:26 Message: Logged In: YES user_id=149084 Please try two things: 1. Run IDLE directly on the ubuntu machine long enough to assure there is no problem standalone. 2. Run IDLE on the xclient but start it with the -n switch so it runs without the subprocess. When IDLE freezes, walk over to the server and kill the subprocess. It should restart itself. Is the system still frozen? And by frozen, I assume you mean all screen and keyboard events are unresponsive. ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-30 20:06 Message: Logged In: YES user_id=993923 Yes, I've used Idle on that (although only ~ 1 hour) and not seen the problem there. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-01-30 19:43 Message: Logged In: YES user_id=341410 Do you have physical access to the ubuntu warty machine, and if so, does Idle run very well locally on that machine, not X forwarded? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-30 17:02 Message: Logged In: YES user_id=993923 Yes, exactly. I'm pretty sure it's not an X/ssh/network problem, everything else works fine (including Idle 0.8). Mark ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-01-29 20:23 Message: Logged In: YES user_id=341410 It has been a while since I ran Idle, but I am curious as to what you mean by "run over ssh". Do you mean that you have an SSH tunnel to a remote machine, which forwards X-Windows sessions, and when running Idle over this, it locks up? ---------------------------------------------------------------------- Comment By: Mark Poolman (mgpoolman) Date: 2005-01-26 09:15 Message: Logged In: YES user_id=993923 PS - You don't need an editor window open to get the freeze. M ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1108992&group_id=5470 From noreply at sourceforge.net Tue Apr 19 14:07:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 14:07:18 2005 Subject: [ python-Bugs-1185883 ] PyObject_Realloc bug in obmalloc.c Message-ID: Bugs item #1185883, was opened at 2005-04-19 12:07 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Kristj?n Valur (krisvale) Assigned to: Nobody/Anonymous (nobody) Summary: PyObject_Realloc bug in obmalloc.c Initial Comment: obmalloc.c:835 If the previous block was not handled by obmalloc, and the realloc is for growing the block, this memcpy may cross a page boundary and cause a segmentation fault. This scenario can happen if a previous allocation failed to successfully allocate from the obmalloc pools, due to memory starvation or other reasons, but was successfully allocated by the c runtime. The solution is to query the actual size of the allocated block, and copy only so much memory. Most modern platforms provide size query functions complementing the malloc()/free() calls. on Windows, this is the _msize () function. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 From noreply at sourceforge.net Tue Apr 19 15:01:22 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 15:01:24 2005 Subject: [ python-Bugs-1185931 ] python socketmodule dies on ^c Message-ID: Bugs item #1185931, was opened at 2005-04-19 13:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: nodata (nodata101) Assigned to: Nobody/Anonymous (nobody) Summary: python socketmodule dies on ^c Initial Comment: I'm using yum on FC4T2 to apply updates to my computer. When I press ^c, yum does not exit, but switches mirror. I reported this to the fedora-test-list, and the maintainer of yum believes this problem to be inside the python socketmodule - in the C code. The thread is here: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01372.html Hopefully this is the right place to report the bug. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 From noreply at sourceforge.net Tue Apr 19 15:03:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 15:03:26 2005 Subject: [ python-Bugs-1185931 ] python socketmodule dies on ^c Message-ID: Bugs item #1185931, was opened at 2005-04-19 13:01 Message generated for change (Comment added) made by nodata101 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: nodata (nodata101) Assigned to: Nobody/Anonymous (nobody) Summary: python socketmodule dies on ^c Initial Comment: I'm using yum on FC4T2 to apply updates to my computer. When I press ^c, yum does not exit, but switches mirror. I reported this to the fedora-test-list, and the maintainer of yum believes this problem to be inside the python socketmodule - in the C code. The thread is here: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01372.html Hopefully this is the right place to report the bug. ---------------------------------------------------------------------- >Comment By: nodata (nodata101) Date: 2005-04-19 13:03 Message: Logged In: YES user_id=960750 Correct ref is: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01545.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 From noreply at sourceforge.net Tue Apr 19 16:21:08 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 16:21:10 2005 Subject: [ python-Bugs-1185931 ] python socketmodule dies on ^c Message-ID: Bugs item #1185931, was opened at 2005-04-19 14:01 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: nodata (nodata101) Assigned to: Nobody/Anonymous (nobody) Summary: python socketmodule dies on ^c Initial Comment: I'm using yum on FC4T2 to apply updates to my computer. When I press ^c, yum does not exit, but switches mirror. I reported this to the fedora-test-list, and the maintainer of yum believes this problem to be inside the python socketmodule - in the C code. The thread is here: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01372.html Hopefully this is the right place to report the bug. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-19 15:21 Message: Logged In: YES user_id=6656 Maybe you can persuade the yum maintainers to tell us what the problem actually is? I don't see anything useful in that thread, and don't particularly want to read the yum sources to find out. ---------------------------------------------------------------------- Comment By: nodata (nodata101) Date: 2005-04-19 14:03 Message: Logged In: YES user_id=960750 Correct ref is: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01545.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 From noreply at sourceforge.net Tue Apr 19 16:30:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 16:30:56 2005 Subject: [ python-Bugs-1185883 ] PyObject_Realloc bug in obmalloc.c Message-ID: Bugs item #1185883, was opened at 2005-04-19 13:07 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Kristj?n Valur (krisvale) >Assigned to: Tim Peters (tim_one) Summary: PyObject_Realloc bug in obmalloc.c Initial Comment: obmalloc.c:835 If the previous block was not handled by obmalloc, and the realloc is for growing the block, this memcpy may cross a page boundary and cause a segmentation fault. This scenario can happen if a previous allocation failed to successfully allocate from the obmalloc pools, due to memory starvation or other reasons, but was successfully allocated by the c runtime. The solution is to query the actual size of the allocated block, and copy only so much memory. Most modern platforms provide size query functions complementing the malloc()/free() calls. on Windows, this is the _msize () function. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-19 15:30 Message: Logged In: YES user_id=6656 Tim, what do you think? This is a pretty unlikely scenario, it seems to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 From noreply at sourceforge.net Tue Apr 19 16:34:25 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 16:34:29 2005 Subject: [ python-Bugs-1181619 ] Bad sys.executable value for bdist_wininst install script Message-ID: Bugs item #1181619, was opened at 2005-04-12 17:49 Message generated for change (Settings changed) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181619&group_id=5470 Category: Distutils Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: follower (xfollower) >Assigned to: Thomas Heller (theller) Summary: Bad sys.executable value for bdist_wininst install script Initial Comment: Description copied from: >From the Python docs, sys.executable is: executable A string giving the name of the executable binary for the Python interpreter, on systems where this makes sense. However, during the execution of a post-install script, this string actually resolves to the name of the binary installer! This name should resolve, I think to the name of the Python executable for which the installer is running (a value selectable at the start of the installation, if more than one Python is detected). Having this value available allows you to properly generate shortcuts with the proper full path to the python executable. I resorted to using sys.prefix+r'\python.exe', which will most likely work, but I'd rather see sys.executable give me a more sensible answer. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181619&group_id=5470 From noreply at sourceforge.net Tue Apr 19 16:37:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 16:37:53 2005 Subject: [ python-Bugs-1175967 ] StringIO and cStringIO don't provide 'name' attribute Message-ID: Bugs item #1175967, was opened at 2005-04-03 21:20 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175967&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: logistix (logistix) Assigned to: Nobody/Anonymous (nobody) Summary: StringIO and cStringIO don't provide 'name' attribute Initial Comment: Documentation explicitly states that file-like objects should return a repr-style psuedoname. Patch is attached. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-19 15:37 Message: Logged In: YES user_id=6656 Agree with Just here. ---------------------------------------------------------------------- Comment By: Just van Rossum (jvr) Date: 2005-04-07 09:54 Message: Logged In: YES user_id=92689 The documentation also says "This is a read-only attribute and may not be present on all file-like objects.", so I'm inclined to close as "won't fix". I'm sure many in-the-wild file-like objects don't support it, either, so depending on its existence is bad style at best. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175967&group_id=5470 From noreply at sourceforge.net Tue Apr 19 16:39:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 16:39:08 2005 Subject: [ python-Bugs-1185883 ] PyObject_Realloc bug in obmalloc.c Message-ID: Bugs item #1185883, was opened at 2005-04-19 12:07 Message generated for change (Comment added) made by krisvale You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Kristj?n Valur (krisvale) Assigned to: Tim Peters (tim_one) Summary: PyObject_Realloc bug in obmalloc.c Initial Comment: obmalloc.c:835 If the previous block was not handled by obmalloc, and the realloc is for growing the block, this memcpy may cross a page boundary and cause a segmentation fault. This scenario can happen if a previous allocation failed to successfully allocate from the obmalloc pools, due to memory starvation or other reasons, but was successfully allocated by the c runtime. The solution is to query the actual size of the allocated block, and copy only so much memory. Most modern platforms provide size query functions complementing the malloc()/free() calls. on Windows, this is the _msize () function. ---------------------------------------------------------------------- >Comment By: Kristj?n Valur (krisvale) Date: 2005-04-19 14:39 Message: Logged In: YES user_id=1262199 I can only say that I?ve been seeing this happeing with our software. Admittedly it's because we are eating up all memory due to other reasons, but we would like to deal with that with a MemoryError rather than a crash. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 14:30 Message: Logged In: YES user_id=6656 Tim, what do you think? This is a pretty unlikely scenario, it seems to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 From noreply at sourceforge.net Tue Apr 19 16:58:01 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 16:58:04 2005 Subject: [ python-Bugs-1122301 ] marshal may crash on truncated input Message-ID: Bugs item #1122301, was opened at 2005-02-14 11:14 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1122301&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fredrik Lundh (effbot) >Assigned to: Fredrik Lundh (effbot) Summary: marshal may crash on truncated input Initial Comment: marshal doesn't behave well on truncated or otherwise malformed input. here's a short demo script, from a recent comp.lang.python thread: ::: the problem is that the following may or may not reach the "done!" statement, somewhat depending on python version, memory allocator, and what data you pass to dumps. import marshal data = marshal.dumps((1, 2, 3, "hello", 4, 5, 6)) for i in range(len(data), -1, -1): try: print marshal.loads(data[:i]) except EOFError: print "EOFError" except ValueError: print "ValueError" print "done!" (try different data combinations, to see how far you get on your platform...) fixing this should be relatively easy, and should result in a safe unmarshaller (your application will still have to limit the amount of data fed into load/loads, of course). ::: (also note that marshal may raise either EOFError or ValueError exceptions, again somewhat depending on how the file is damaged. a little consistency wouldn't hurt, but I'm not sure if/how this can be fixed...) ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-19 15:58 Message: Logged In: YES user_id=6656 I think the attached fixes this example, and another involving marshalled sets. I spent a while feeding random data to marshal a few days ago and found that the commonest problem was attempting to allocate really huge sequences. Also, the TYPE_STRINGREF is horribly fragile, but I'm hoping Martin's going to fix that (he has a bug filed against him, anyway). Can you test/check it in? My marshal.c has rather a lot of local changes. Also, a test suite entry would be nice... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1122301&group_id=5470 From noreply at sourceforge.net Tue Apr 19 17:00:39 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 17:00:41 2005 Subject: [ python-Bugs-1185883 ] PyObject_Realloc bug in obmalloc.c Message-ID: Bugs item #1185883, was opened at 2005-04-19 08:07 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Kristj?n Valur (krisvale) >Assigned to: Nobody/Anonymous (nobody) Summary: PyObject_Realloc bug in obmalloc.c Initial Comment: obmalloc.c:835 If the previous block was not handled by obmalloc, and the realloc is for growing the block, this memcpy may cross a page boundary and cause a segmentation fault. This scenario can happen if a previous allocation failed to successfully allocate from the obmalloc pools, due to memory starvation or other reasons, but was successfully allocated by the c runtime. The solution is to query the actual size of the allocated block, and copy only so much memory. Most modern platforms provide size query functions complementing the malloc()/free() calls. on Windows, this is the _msize () function. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-19 11:00 Message: Logged In: YES user_id=31435 mwh: Umm ... I don't understand what the claim is. For example, what HW does Python run on where memcpy segfaults just because the address range crosses a page boundary? If that's what's happening, sounds more like a bug in the platform memcpy. I can memcpy blocks spanning thousands of pages on my box -- and so can you . krisvale: which OS and which C are you using? It is true that this code may try to access a bit of memory that wasn't allocated. If that's at the end of the address space, then I could see a segfault happening. If it is, I doubt there's any portable way to fix it short of PyObject_Realloc never trying to take over small blocks it didn't control to begin with. Then the platform realloc() will segfault instead . ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2005-04-19 10:39 Message: Logged In: YES user_id=1262199 I can only say that I?ve been seeing this happeing with our software. Admittedly it's because we are eating up all memory due to other reasons, but we would like to deal with that with a MemoryError rather than a crash. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 10:30 Message: Logged In: YES user_id=6656 Tim, what do you think? This is a pretty unlikely scenario, it seems to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 From noreply at sourceforge.net Tue Apr 19 17:02:42 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 17:02:48 2005 Subject: [ python-Bugs-1100673 ] Python Interpreter shell is crashed Message-ID: Bugs item #1100673, was opened at 2005-01-12 05:49 Message generated for change (Settings changed) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1100673&group_id=5470 Category: Python Interpreter Core Group: Python 2.2.2 >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: abhishek (abhishekkabra) Assigned to: Nobody/Anonymous (nobody) Summary: Python Interpreter shell is crashed Initial Comment: I faced this problem about 50 % of time when I hit follwing commands on python shell. But I think Crash of interpreter is not a expected behaviour. It should throw some error even if I am wrong/ hitting wrong commands 1. on shell of linux start python 2 On python shell hit _doc__ ( underscore doc underscore underscore ) So Python shell is crashed with following crash dump darwin{akabra}6: python Python 2.2.2 (#1, Feb 24 2003, 19:13:11) [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> _doc__Segmentation fault (core dumped) darwin{akabra}7: ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-19 16:02 Message: Logged In: YES user_id=6656 Closing for want of activity. ---------------------------------------------------------------------- Comment By: Facundo Batista (facundobatista) Date: 2005-01-15 20:55 Message: Logged In: YES user_id=752496 Works ok to me. ---------------------------------------------------------------------- Comment By: Facundo Batista (facundobatista) Date: 2005-01-15 20:55 Message: Logged In: YES user_id=752496 Please, could you verify if this problem persists in Python 2.3.4 or 2.4? If yes, in which version? Can you provide a test case? If the problem is solved, from which version? Note that if you fail to answer in one month, I'll close this bug as "Won't fix". Thank you! . Facundo ---------------------------------------------------------------------- Comment By: Puneet (puneet_mnitian) Date: 2005-01-13 05:46 Message: Logged In: YES user_id=1196195 Not reproducible ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-01-12 19:16 Message: Logged In: YES user_id=6656 That's certainly not expected behaviour, however I think it's unlikely to be a python bug -- I've not heard of this behaviour before. Is python using readline? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1100673&group_id=5470 From noreply at sourceforge.net Tue Apr 19 17:14:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 17:14:26 2005 Subject: [ python-Bugs-1185931 ] python socketmodule dies on ^c Message-ID: Bugs item #1185931, was opened at 2005-04-19 13:01 Message generated for change (Comment added) made by nodata101 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: nodata (nodata101) Assigned to: Nobody/Anonymous (nobody) Summary: python socketmodule dies on ^c Initial Comment: I'm using yum on FC4T2 to apply updates to my computer. When I press ^c, yum does not exit, but switches mirror. I reported this to the fedora-test-list, and the maintainer of yum believes this problem to be inside the python socketmodule - in the C code. The thread is here: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01372.html Hopefully this is the right place to report the bug. ---------------------------------------------------------------------- >Comment By: nodata (nodata101) Date: 2005-04-19 15:14 Message: Logged In: YES user_id=960750 Sorry - already reported upstream by yum maintainer. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 14:21 Message: Logged In: YES user_id=6656 Maybe you can persuade the yum maintainers to tell us what the problem actually is? I don't see anything useful in that thread, and don't particularly want to read the yum sources to find out. ---------------------------------------------------------------------- Comment By: nodata (nodata101) Date: 2005-04-19 13:03 Message: Logged In: YES user_id=960750 Correct ref is: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01545.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 From noreply at sourceforge.net Tue Apr 19 17:20:30 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 17:20:33 2005 Subject: [ python-Bugs-1185931 ] python socketmodule dies on ^c Message-ID: Bugs item #1185931, was opened at 2005-04-19 14:01 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 Category: Extension Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: nodata (nodata101) Assigned to: Nobody/Anonymous (nobody) Summary: python socketmodule dies on ^c Initial Comment: I'm using yum on FC4T2 to apply updates to my computer. When I press ^c, yum does not exit, but switches mirror. I reported this to the fedora-test-list, and the maintainer of yum believes this problem to be inside the python socketmodule - in the C code. The thread is here: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01372.html Hopefully this is the right place to report the bug. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-19 16:20 Message: Logged In: YES user_id=6656 Huh? Where? Should this be closed as a duplicate? ---------------------------------------------------------------------- Comment By: nodata (nodata101) Date: 2005-04-19 16:14 Message: Logged In: YES user_id=960750 Sorry - already reported upstream by yum maintainer. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 15:21 Message: Logged In: YES user_id=6656 Maybe you can persuade the yum maintainers to tell us what the problem actually is? I don't see anything useful in that thread, and don't particularly want to read the yum sources to find out. ---------------------------------------------------------------------- Comment By: nodata (nodata101) Date: 2005-04-19 14:03 Message: Logged In: YES user_id=960750 Correct ref is: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01545.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 From noreply at sourceforge.net Tue Apr 19 17:22:26 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 17:22:29 2005 Subject: [ python-Bugs-1185883 ] PyObject_Realloc bug in obmalloc.c Message-ID: Bugs item #1185883, was opened at 2005-04-19 12:07 Message generated for change (Comment added) made by krisvale You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Kristj?n Valur (krisvale) Assigned to: Nobody/Anonymous (nobody) Summary: PyObject_Realloc bug in obmalloc.c Initial Comment: obmalloc.c:835 If the previous block was not handled by obmalloc, and the realloc is for growing the block, this memcpy may cross a page boundary and cause a segmentation fault. This scenario can happen if a previous allocation failed to successfully allocate from the obmalloc pools, due to memory starvation or other reasons, but was successfully allocated by the c runtime. The solution is to query the actual size of the allocated block, and copy only so much memory. Most modern platforms provide size query functions complementing the malloc()/free() calls. on Windows, this is the _msize () function. ---------------------------------------------------------------------- >Comment By: Kristj?n Valur (krisvale) Date: 2005-04-19 15:22 Message: Logged In: YES user_id=1262199 The platform is windows 2000/2003 server, single threaded C runtime. I have only had the chance to do postmortem debugging on this but it would appear to be as you describe: The following page is not mapped in. Windows doesn?t use the setbrk() method of heap management and doesn?t automatically move the break. Rather they (the multiple heaps) requests pages as required. A malloc may have succeeded from a different page and copying to much from the old block close to the boundary caused an exception _at_ the page boundary. Fyi, old block was 68 bytes at 0x6d85efb8. This block ends at -effc. The new size requested was 108 bytes. Reading 108 bytes from this address caused an exception at address 0x6d85f000. As you know, reading past a malloc block results in undefined behaviour and sometimes this can mean a crash. I have patched python locally to use MIN(nbytes, _msize(p)) in stead and we are about to run the modified version on our server cluster. Nodes were dying quite regularly because of this. I'll let you know if this changes anyting in that aspect. Btw, I work for ccp games, and we are running the MMORPG eve online (www.eveonline.com) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-19 15:00 Message: Logged In: YES user_id=31435 mwh: Umm ... I don't understand what the claim is. For example, what HW does Python run on where memcpy segfaults just because the address range crosses a page boundary? If that's what's happening, sounds more like a bug in the platform memcpy. I can memcpy blocks spanning thousands of pages on my box -- and so can you . krisvale: which OS and which C are you using? It is true that this code may try to access a bit of memory that wasn't allocated. If that's at the end of the address space, then I could see a segfault happening. If it is, I doubt there's any portable way to fix it short of PyObject_Realloc never trying to take over small blocks it didn't control to begin with. Then the platform realloc() will segfault instead . ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2005-04-19 14:39 Message: Logged In: YES user_id=1262199 I can only say that I?ve been seeing this happeing with our software. Admittedly it's because we are eating up all memory due to other reasons, but we would like to deal with that with a MemoryError rather than a crash. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 14:30 Message: Logged In: YES user_id=6656 Tim, what do you think? This is a pretty unlikely scenario, it seems to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 From noreply at sourceforge.net Tue Apr 19 17:34:50 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 17:34:52 2005 Subject: [ python-Bugs-1185883 ] PyObject_Realloc bug in obmalloc.c Message-ID: Bugs item #1185883, was opened at 2005-04-19 08:07 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 Category: Python Interpreter Core Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Kristj?n Valur (krisvale) >Assigned to: Tim Peters (tim_one) Summary: PyObject_Realloc bug in obmalloc.c Initial Comment: obmalloc.c:835 If the previous block was not handled by obmalloc, and the realloc is for growing the block, this memcpy may cross a page boundary and cause a segmentation fault. This scenario can happen if a previous allocation failed to successfully allocate from the obmalloc pools, due to memory starvation or other reasons, but was successfully allocated by the c runtime. The solution is to query the actual size of the allocated block, and copy only so much memory. Most modern platforms provide size query functions complementing the malloc()/free() calls. on Windows, this is the _msize () function. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-19 11:34 Message: Logged In: YES user_id=31435 krisvale: Thank you for the very clear explanation. Even I understand this now . We won't use _msize here -- Python has to run under dozens of compilers and C libraries, and it's saner to give up on this "optimization" completely than to introduce a rat's nest of #ifdefs here. IOW, I expect the entire "if (nbytes <= SMALL_REQUEST_THRESHOLD)" block will go away, so that the platform realloc() gets called in every case obmalloc doesn't control the incoming block. BTW, note that there's no plan to do another release in the Python 2.3 line. ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2005-04-19 11:22 Message: Logged In: YES user_id=1262199 The platform is windows 2000/2003 server, single threaded C runtime. I have only had the chance to do postmortem debugging on this but it would appear to be as you describe: The following page is not mapped in. Windows doesn?t use the setbrk() method of heap management and doesn?t automatically move the break. Rather they (the multiple heaps) requests pages as required. A malloc may have succeeded from a different page and copying to much from the old block close to the boundary caused an exception _at_ the page boundary. Fyi, old block was 68 bytes at 0x6d85efb8. This block ends at -effc. The new size requested was 108 bytes. Reading 108 bytes from this address caused an exception at address 0x6d85f000. As you know, reading past a malloc block results in undefined behaviour and sometimes this can mean a crash. I have patched python locally to use MIN(nbytes, _msize(p)) in stead and we are about to run the modified version on our server cluster. Nodes were dying quite regularly because of this. I'll let you know if this changes anyting in that aspect. Btw, I work for ccp games, and we are running the MMORPG eve online (www.eveonline.com) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-19 11:00 Message: Logged In: YES user_id=31435 mwh: Umm ... I don't understand what the claim is. For example, what HW does Python run on where memcpy segfaults just because the address range crosses a page boundary? If that's what's happening, sounds more like a bug in the platform memcpy. I can memcpy blocks spanning thousands of pages on my box -- and so can you . krisvale: which OS and which C are you using? It is true that this code may try to access a bit of memory that wasn't allocated. If that's at the end of the address space, then I could see a segfault happening. If it is, I doubt there's any portable way to fix it short of PyObject_Realloc never trying to take over small blocks it didn't control to begin with. Then the platform realloc() will segfault instead . ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2005-04-19 10:39 Message: Logged In: YES user_id=1262199 I can only say that I?ve been seeing this happeing with our software. Admittedly it's because we are eating up all memory due to other reasons, but we would like to deal with that with a MemoryError rather than a crash. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 10:30 Message: Logged In: YES user_id=6656 Tim, what do you think? This is a pretty unlikely scenario, it seems to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185883&group_id=5470 From noreply at sourceforge.net Tue Apr 19 17:49:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 17:49:20 2005 Subject: [ python-Bugs-1186072 ] tempnam doc doesn't include link to tmpfile Message-ID: Bugs item #1186072, was opened at 2005-04-19 10:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186072&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ian Bicking (ianbicking) Assigned to: Nobody/Anonymous (nobody) Summary: tempnam doc doesn't include link to tmpfile Initial Comment: Both tmpnam and tempnam include references to tmpfile (as a preferred way of using temporary files). However, they don't include a link to the page where tmpfile is documented, and it is documented in a different (non-obvious) section of the ``os`` page. A link to the section containing tmpfile would be helpful. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186072&group_id=5470 From noreply at sourceforge.net Tue Apr 19 18:45:05 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 18:45:08 2005 Subject: [ python-Bugs-548661 ] os.popen w/o using the shell Message-ID: Bugs item #548661, was opened at 2002-04-25 10:30 Message generated for change (Comment added) made by ianbicking You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=548661&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Ian Bicking (ianbicking) Assigned to: Nobody/Anonymous (nobody) Summary: os.popen w/o using the shell Initial Comment: I heard that there was an undocumented feature to the os.popen family, where instead of passing a command string as the first argument you could pass a list, as [path, arg1, arg2, ...], and circumvent any shell interpretation. I was disapointed to see that this was not so ("popen() argument 1 must be string, not list" ditto tuple) I believe this would be an excellent feature -- using the shell is a significant source of errors due to quoting, as well as a serious security concern. 95% of the time the shell is not required. The shell also introduces portability concerns (e.g., bug #512433) -- creating a Python shell is not necessary when the shell is usually superfluous anyway. ---------------------------------------------------------------------- >Comment By: Ian Bicking (ianbicking) Date: 2005-04-19 11:45 Message: Logged In: YES user_id=210337 This may not matter enough to resolve now, with the advent of the subprocess module. ---------------------------------------------------------------------- Comment By: Peter ?strand (astrand) Date: 2003-11-01 10:29 Message: Logged In: YES user_id=344921 popen5 never uses the shell (http://www.lysator.liu.se/~astrand/popen5/) ---------------------------------------------------------------------- Comment By: Juli?n Mu?oz (julian69) Date: 2003-01-08 10:15 Message: Logged In: YES user_id=77756 Does this mean that giving a list to popen2 free us from taking care of the dangerous characters that could be interprated/escaped by the shell ??? I don't find any documentation about this feature !!! ---------------------------------------------------------------------- Comment By: Ian Bicking (ianbicking) Date: 2002-04-25 13:18 Message: Logged In: YES user_id=210337 I see you are correct. It would be nice if this feature was consistent across all popen*, and was also documented (and so also committed to with clear semantics) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2002-04-25 13:13 Message: Logged In: YES user_id=21627 This feature is indeed available in popen2. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=548661&group_id=5470 From noreply at sourceforge.net Tue Apr 19 18:53:26 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 18:53:31 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 21:03 Message generated for change (Comment added) made by lcaamano You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 12:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 03:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 23:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-05 21:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Tue Apr 19 19:11:27 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 19:11:31 2005 Subject: [ python-Bugs-1185124 ] pydoc doesn't find all module doc strings Message-ID: Bugs item #1185124, was opened at 2005-04-18 08:18 Message generated for change (Comment added) made by brianvanden You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Kent Johnson (kjohnson) Assigned to: Ka-Ping Yee (ping) Summary: pydoc doesn't find all module doc strings Initial Comment: pydoc.synopsis() attempts to find a module's doc string by parsing the module text. But the parser only recognizes strings created with """ and r""". Any other docstring is ignored. I've attached a patch against Python 2.4.1 that fixes pydoc to recognize ''' and r''' strings but really it should recognize any allowable string format. ---------------------------------------------------------------------- Comment By: Brian vdB (brianvanden) Date: 2005-04-19 13:11 Message: Logged In: YES user_id=1015686 I started the thread to which Kent referred. I am aware of PEP 257's recommendation of triple-double quotes. My (perhaps wrong-headed) construal of that PEP is that it isn't sufficiently rule-giving that I would have expected other tools to reject triple-single quotes. At any rate, since triple-single are syntactically acceptable, it would seem better if they were accepted on equal footing with triple-double. I can well understand that this would be a v. low priority issue, though. Call it a RFE. :-) ---------------------------------------------------------------------- Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 16:28 Message: Logged In: YES user_id=45338 I think you're right that if it works for the module summary (using __doc__) then it should work with synopsis(). However, the patch you've added doesn't address the problem properly; instead of handling """ correctly and ignoring ''', it handles both kinds of docstrings incorrectly because it will accept ''' as a match for """ or """ as a match for '''. I'll look at fixing this soon, but feel free to keep prodding me until it gets fixed. ---------------------------------------------------------------------- Comment By: Kent Johnson (kjohnson) Date: 2005-04-18 16:04 Message: Logged In: YES user_id=49695 I don't know if there are a large number of modules with triple-single-quoted docstrings. Pydoc will search any module in site-packages at least, so you have to consider third-party modules. At best pydoc is inconsistent - the web browser display uses the __doc__attribute but search and apropos use synopsis(). It's a pretty simple change to recognize any triple-quoted string, it seems like a good idea to me... I have attached a revised patch that uses a regex match so it works with e.g. uR""" and other variations of triple-quoting. FWIW this bug report was motivated by this thread on comp.lang.python: http://groups-beta.google.com/group/comp.lang.python/browse_frm/thread/e5cfccb7c9a168d7/1c1702e71e1939b0?q=triple&rnum=1#1c1702e71e1939b0 ---------------------------------------------------------------------- Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 14:23 Message: Logged In: YES user_id=45338 PEP 257 recommends: "For consistency, always use """triple double quotes""" around docstrings." I think that's why this was originally written to only look for triple double-quotes. Are there a large number of modules written using triple-single quotes for the module docstring? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185124&group_id=5470 From noreply at sourceforge.net Tue Apr 19 19:49:55 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 19:50:00 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 18:03 Message generated for change (Comment added) made by majid You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Fazal Majid (majid) Date: 2005-04-19 10:49 Message: Logged In: YES user_id=110477 Modifying the system os.py is not a good idea. A better work-around is to skip the /dev/urandom fd when you are closing all fds. This is the code we use: def close_fd(): # close all inherited file descriptors start_fd = 3 # Python 2.4.1 and later use /dev/urandom to seed the random module's RNG # a file descriptor is kept internally as os._urandomfd (created on demand # the first time os.urandom() is called), and should not be closed try: os.urandom(4) urandom_fd = getattr(os, '_urandomfd', None) except AttributeError: urandom_fd = None if '-close_fd' in sys.argv: start_fd = int(sys.argv[sys.argv.index('-close_fd') + 1]) for fd in range(start_fd, 256): if fd == urandom_fd: continue try: os.close(fd) except OSError: pass ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 09:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 02:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 00:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 00:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 00:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 20:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-05 18:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Tue Apr 19 20:58:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 20:58:21 2005 Subject: [ python-Feature Requests-1185121 ] itertools.imerge: merge sequences Message-ID: Feature Requests item #1185121, was opened at 2005-04-18 07:11 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jurjen N.E. Bos (jneb) Assigned to: Raymond Hettinger (rhettinger) Summary: itertools.imerge: merge sequences Initial Comment: (For the itertools library, so Python 2.2 and up) This is a suggested addition to itertools, proposed name imerge. usage: imerge(seq0, seq1, ..., [key=]) result: imerge assumes the sequences are all in sorted order, and produces a iterator that returns pairs of the form (value, index), where value is a value of one of the sequences, and index is the index number of the given sequence. The output the imerge is in sorted order (taking into account the key function), so that identical values in the sequences will be produced from left to right. The code is surprisingly short, making use of the builtin heap module. (You may disagree with my style of argument handling; feel free to optimize it.) def imerge(*iterlist, **key): """Merge a sequence of sorted iterables. Returns pairs [value, index] where each value comes from iterlist[index], and the pairs are sorted if each of the iterators is sorted. Hint use groupby(imerge(...), operator.itemgetter(0)) to get the items one by one. """ if key.keys() not in ([], ["key"]): raise TypeError, "Excess keyword arguments for imerge" key = key.get("key", lambda x:x) from heapq import heapreplace, heappop #initialize the heap containing (inited, value, index, currentItem, iterator) #this automatically makes sure all iterators are initialized, then run, and finally emptied heap = [(False, None, index, None, iter(iterator)) for index, iterator in enumerate(iterlist)] while heap: inited, item, index, value, iterator = heap[0] if inited: yield value, index try: item = iterator.next() except StopIteration: heappop(heap) else: heapreplace(heap, (True, key(item), index, item, iterator)) If you find this little routine worth its size, please put it into itertools. - Jurjen ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 13:58 Message: Logged In: YES user_id=80475 For your specific application, it is better to use sorted(). When the underlying data consists of long runs of previously ordered data, sorted() will take advantage of that ordering and run in O(n) time. In contrast, using a heap will unnecessarily introduce O(n log n) behavior and not exploit the underlying data order. Recommend that you close this request. This discussion thus far confirms the original conclusion that imerge() use cases are dominated by sorted(chain(*iterlist)) which gives code that is shorter, faster, and easier to understand. ---------------------------------------------------------------------- Comment By: Jurjen N.E. Bos (jneb) Date: 2005-04-19 03:19 Message: Logged In: YES user_id=446428 Well, I was optimizing a piece of code with reasonbly long sorted lists (in memory, I agree) that were modified in all kinds of ways. I did not want the nlogn behaviour of sort, so I started writing a merge routine. I found out that the boundary cases of a merge implementation are a mess, until I disccovered the heap trick. Then I decided to clean it up and and put it up for a library routine. The fact that it uses iterators is obnly to make it more general, not specifically for the "lazy" properties. - Jurjen ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:43 Message: Logged In: YES user_id=80475 I had previously looked at an imerge() utility and found that it had only a single application (isomorphic to lazy mergesorting) and that the use cases were dominated by the in-memory alternative: sorted(chain(*iterlist)). Short of writing an external mergesort, what applications did you have in mind? What situations have you encountered where you have multiple sources of sorted data being generated on the fly (as opposed to already being in-memory), have needed one element at a time sequential access to a combined sort of that data, needed that combined sort only once, and could not afford to have the dataset in-memory? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 From noreply at sourceforge.net Tue Apr 19 21:02:15 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 21:02:18 2005 Subject: [ python-Bugs-1186195 ] [AST] genexps get scoping wrong Message-ID: Bugs item #1186195, was opened at 2005-04-19 12:02 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nick Coghlan (ncoghlan) Summary: [AST] genexps get scoping wrong Initial Comment: test_genexps is failing because it is unable to find a global defined in a genexp that is returned. Here is the problem simplified: def f(n): return (i for i in xrange(n)) list(f(10)) Leads to ``SystemError: no locals when loading 'xrange'``. Comparing Python 2.4 bytecode: 1 0 LOAD_CONST 1 ( at 0x3931e0, file "", line 1>) 3 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (xrange) 9 LOAD_FAST 0 (n) 12 CALL_FUNCTION 1 15 GET_ITER 16 CALL_FUNCTION 1 19 RETURN_VALUE 20 LOAD_CONST 0 (None) 23 RETURN_VALUE to AST bytecode: 1 0 LOAD_CLOSURE 0 (n) 3 BUILD_TUPLE 1 6 LOAD_CONST 1 ( at 0x5212e8, file "", line 1>) 9 MAKE_CLOSURE 0 12 LOAD_NAME 0 (xrange) 15 LOAD_DEREF 0 (n) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 RETURN_VALUE 26 LOAD_CONST 0 (None) 29 RETURN_VALUE makes it obvious something is off (no peepholer; turned it off in my build of 2.4). Looks like extraneous work is being done in making a closure. Seems like it will still work, though. Plus the usage of LOAD_NAME is wrong in the AST; LOAD_NAME gets an object from the local namespace based on its name instead of offset. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 From noreply at sourceforge.net Tue Apr 19 21:03:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 21:03:53 2005 Subject: [ python-Bugs-1186195 ] [AST] genexps get scoping wrong Message-ID: Bugs item #1186195, was opened at 2005-04-19 12:02 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nick Coghlan (ncoghlan) Summary: [AST] genexps get scoping wrong Initial Comment: test_genexps is failing because it is unable to find a global defined in a genexp that is returned. Here is the problem simplified: def f(n): return (i for i in xrange(n)) list(f(10)) Leads to ``SystemError: no locals when loading 'xrange'``. Comparing Python 2.4 bytecode: 1 0 LOAD_CONST 1 ( at 0x3931e0, file "", line 1>) 3 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (xrange) 9 LOAD_FAST 0 (n) 12 CALL_FUNCTION 1 15 GET_ITER 16 CALL_FUNCTION 1 19 RETURN_VALUE 20 LOAD_CONST 0 (None) 23 RETURN_VALUE to AST bytecode: 1 0 LOAD_CLOSURE 0 (n) 3 BUILD_TUPLE 1 6 LOAD_CONST 1 ( at 0x5212e8, file "", line 1>) 9 MAKE_CLOSURE 0 12 LOAD_NAME 0 (xrange) 15 LOAD_DEREF 0 (n) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 RETURN_VALUE 26 LOAD_CONST 0 (None) 29 RETURN_VALUE makes it obvious something is off (no peepholer; turned it off in my build of 2.4). Looks like extraneous work is being done in making a closure. Seems like it will still work, though. Plus the usage of LOAD_NAME is wrong in the AST; LOAD_NAME gets an object from the local namespace based on its name instead of offset. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-19 12:03 Message: Logged In: YES user_id=357491 Initially assigned to Nick since he did the genexp patch and in hopes he might know what is going on off the top of his head. Otherwise assign to me. I have a sneaking suspicion that the symtable code overall is slightly busted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 From noreply at sourceforge.net Tue Apr 19 21:46:29 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 21:46:32 2005 Subject: [ python-Bugs-1181619 ] Bad sys.executable value for bdist_wininst install script Message-ID: Bugs item #1181619, was opened at 2005-04-12 18:49 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181619&group_id=5470 Category: Distutils Group: Platform-specific >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: follower (xfollower) Assigned to: Thomas Heller (theller) Summary: Bad sys.executable value for bdist_wininst install script Initial Comment: Description copied from: >From the Python docs, sys.executable is: executable A string giving the name of the executable binary for the Python interpreter, on systems where this makes sense. However, during the execution of a post-install script, this string actually resolves to the name of the binary installer! This name should resolve, I think to the name of the Python executable for which the installer is running (a value selectable at the start of the installation, if more than one Python is detected). Having this value available allows you to properly generate shortcuts with the proper full path to the python executable. I resorted to using sys.prefix+r'\python.exe', which will most likely work, but I'd rather see sys.executable give me a more sensible answer. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2005-04-19 21:46 Message: Logged In: YES user_id=11105 I interpreted the docs you quote as 'the Python interpreter that is currently running' and not 'the Python interpreter that is normally used'. OTOH, it's too late to change this for 2.4 anyway. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1181619&group_id=5470 From noreply at sourceforge.net Tue Apr 19 22:05:26 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 22:05:31 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 21:03 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-19 16:05 Message: Logged In: YES user_id=6380 I recommend to close this as invalid. The daemonization code is clearly broken. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-19 13:49 Message: Logged In: YES user_id=110477 Modifying the system os.py is not a good idea. A better work-around is to skip the /dev/urandom fd when you are closing all fds. This is the code we use: def close_fd(): # close all inherited file descriptors start_fd = 3 # Python 2.4.1 and later use /dev/urandom to seed the random module's RNG # a file descriptor is kept internally as os._urandomfd (created on demand # the first time os.urandom() is called), and should not be closed try: os.urandom(4) urandom_fd = getattr(os, '_urandomfd', None) except AttributeError: urandom_fd = None if '-close_fd' in sys.argv: start_fd = int(sys.argv[sys.argv.index('-close_fd') + 1]) for fd in range(start_fd, 256): if fd == urandom_fd: continue try: os.close(fd) except OSError: pass ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 12:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 03:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 23:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-05 21:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Tue Apr 19 23:04:09 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 19 23:04:11 2005 Subject: [ python-Bugs-1185931 ] python socketmodule dies on ^c Message-ID: Bugs item #1185931, was opened at 2005-04-19 13:01 Message generated for change (Comment added) made by nodata101 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 Category: Extension Modules Group: Python 2.4 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: nodata (nodata101) Assigned to: Nobody/Anonymous (nobody) Summary: python socketmodule dies on ^c Initial Comment: I'm using yum on FC4T2 to apply updates to my computer. When I press ^c, yum does not exit, but switches mirror. I reported this to the fedora-test-list, and the maintainer of yum believes this problem to be inside the python socketmodule - in the C code. The thread is here: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01372.html Hopefully this is the right place to report the bug. ---------------------------------------------------------------------- >Comment By: nodata (nodata101) Date: 2005-04-19 21:04 Message: Logged In: YES user_id=960750 Closing as dupe. Do not have bug ref no. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 15:20 Message: Logged In: YES user_id=6656 Huh? Where? Should this be closed as a duplicate? ---------------------------------------------------------------------- Comment By: nodata (nodata101) Date: 2005-04-19 15:14 Message: Logged In: YES user_id=960750 Sorry - already reported upstream by yum maintainer. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-19 14:21 Message: Logged In: YES user_id=6656 Maybe you can persuade the yum maintainers to tell us what the problem actually is? I don't see anything useful in that thread, and don't particularly want to read the yum sources to find out. ---------------------------------------------------------------------- Comment By: nodata (nodata101) Date: 2005-04-19 13:03 Message: Logged In: YES user_id=960750 Correct ref is: https://www.redhat.com/archives/fedora-test-list/2005-April/msg01545.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1185931&group_id=5470 From noreply at sourceforge.net Wed Apr 20 00:27:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 00:27:52 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-06 01:03 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:27 Message: Logged In: YES user_id=81797 Perhaps the best way to resolve this would be for the standard library to provide code that either does the daemonize process, or at least does the closing of the sockets that may be done as part of the daemonize, that way it's clear what the "right" way is to do it. Thoughts? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-19 20:05 Message: Logged In: YES user_id=6380 I recommend to close this as invalid. The daemonization code is clearly broken. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-19 17:49 Message: Logged In: YES user_id=110477 Modifying the system os.py is not a good idea. A better work-around is to skip the /dev/urandom fd when you are closing all fds. This is the code we use: def close_fd(): # close all inherited file descriptors start_fd = 3 # Python 2.4.1 and later use /dev/urandom to seed the random module's RNG # a file descriptor is kept internally as os._urandomfd (created on demand # the first time os.urandom() is called), and should not be closed try: os.urandom(4) urandom_fd = getattr(os, '_urandomfd', None) except AttributeError: urandom_fd = None if '-close_fd' in sys.argv: start_fd = int(sys.argv[sys.argv.index('-close_fd') + 1]) for fd in range(start_fd, 256): if fd == urandom_fd: continue try: os.close(fd) except OSError: pass ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 16:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 09:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 07:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 07:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 07:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 01:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 20 01:24:13 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 01:24:15 2005 Subject: [ python-Bugs-1186345 ] [AST] assert failure on ``eval("u'\Ufffffffe'")`` Message-ID: Bugs item #1186345, was opened at 2005-04-19 16:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] assert failure on ``eval("u'\Ufffffffe'")`` Initial Comment: Isolated the failure of test_unicode to be because of the test of ``eval("u'\Ufffffffe'")``. What is odd is that the Unicode string works fine as a literal at the intepreter prompt. Somehow eval() is triggering this problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 From noreply at sourceforge.net Wed Apr 20 01:37:29 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 01:37:31 2005 Subject: [ python-Bugs-1186353 ] [AST] automatic unpacking of arguments broken Message-ID: Bugs item #1186353, was opened at 2005-04-19 16:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186353&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] automatic unpacking of arguments broken Initial Comment: The code ``(lambda (x, y): x)((3, 5))`` fails because the passed-in tuple is not unpacked into the arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186353&group_id=5470 From noreply at sourceforge.net Wed Apr 20 02:09:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 02:09:44 2005 Subject: [ python-Bugs-1186195 ] [AST] genexps get scoping wrong Message-ID: Bugs item #1186195, was opened at 2005-04-19 12:02 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nick Coghlan (ncoghlan) Summary: [AST] genexps get scoping wrong Initial Comment: test_genexps is failing because it is unable to find a global defined in a genexp that is returned. Here is the problem simplified: def f(n): return (i for i in xrange(n)) list(f(10)) Leads to ``SystemError: no locals when loading 'xrange'``. Comparing Python 2.4 bytecode: 1 0 LOAD_CONST 1 ( at 0x3931e0, file "", line 1>) 3 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (xrange) 9 LOAD_FAST 0 (n) 12 CALL_FUNCTION 1 15 GET_ITER 16 CALL_FUNCTION 1 19 RETURN_VALUE 20 LOAD_CONST 0 (None) 23 RETURN_VALUE to AST bytecode: 1 0 LOAD_CLOSURE 0 (n) 3 BUILD_TUPLE 1 6 LOAD_CONST 1 ( at 0x5212e8, file "", line 1>) 9 MAKE_CLOSURE 0 12 LOAD_NAME 0 (xrange) 15 LOAD_DEREF 0 (n) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 RETURN_VALUE 26 LOAD_CONST 0 (None) 29 RETURN_VALUE makes it obvious something is off (no peepholer; turned it off in my build of 2.4). Looks like extraneous work is being done in making a closure. Seems like it will still work, though. Plus the usage of LOAD_NAME is wrong in the AST; LOAD_NAME gets an object from the local namespace based on its name instead of offset. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-19 17:09 Message: Logged In: YES user_id=357491 Some playing with gdb has turned up some clues. So LOAD_NAME is emitted by compiler_nameop(). How this partially works is that it gets the scope for an argument and then based on that emits the proper load, store, or delete. So why is xrange() coming out at the NAME scope? Well, turns out it is not being found in a particular scope by PyST_GetScope() and is thus returning 0 by default. This means that NAME becomes the scope for xrange(). So it looks like the symtable creation is screwing up and not making xrange() a global like it should. This might be a side-effect of the while closure thing above. Not sure, though. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-19 12:03 Message: Logged In: YES user_id=357491 Initially assigned to Nick since he did the genexp patch and in hopes he might know what is going on off the top of his head. Otherwise assign to me. I have a sneaking suspicion that the symtable code overall is slightly busted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 From noreply at sourceforge.net Wed Apr 20 02:30:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 02:30:48 2005 Subject: [ python-Bugs-1186195 ] [AST] genexps get scoping wrong Message-ID: Bugs item #1186195, was opened at 2005-04-19 12:02 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nick Coghlan (ncoghlan) Summary: [AST] genexps get scoping wrong Initial Comment: test_genexps is failing because it is unable to find a global defined in a genexp that is returned. Here is the problem simplified: def f(n): return (i for i in xrange(n)) list(f(10)) Leads to ``SystemError: no locals when loading 'xrange'``. Comparing Python 2.4 bytecode: 1 0 LOAD_CONST 1 ( at 0x3931e0, file "", line 1>) 3 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (xrange) 9 LOAD_FAST 0 (n) 12 CALL_FUNCTION 1 15 GET_ITER 16 CALL_FUNCTION 1 19 RETURN_VALUE 20 LOAD_CONST 0 (None) 23 RETURN_VALUE to AST bytecode: 1 0 LOAD_CLOSURE 0 (n) 3 BUILD_TUPLE 1 6 LOAD_CONST 1 ( at 0x5212e8, file "", line 1>) 9 MAKE_CLOSURE 0 12 LOAD_NAME 0 (xrange) 15 LOAD_DEREF 0 (n) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 RETURN_VALUE 26 LOAD_CONST 0 (None) 29 RETURN_VALUE makes it obvious something is off (no peepholer; turned it off in my build of 2.4). Looks like extraneous work is being done in making a closure. Seems like it will still work, though. Plus the usage of LOAD_NAME is wrong in the AST; LOAD_NAME gets an object from the local namespace based on its name instead of offset. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-19 17:30 Message: Logged In: YES user_id=357491 OK, figured out why the closure thing is happening. 'n' is being detected as a free variable. This leads to PyCode_GetNumFree() to return a non-0 value. In compiler_make_closure() this automatically triggers the LOAD_CLOSURE/.../MAKE_CLOSURE chunk of bytecode instead of LOAD_CONST/MAKE_FUNCTION. So, how to make 'n' be detected as a local instead a free variable ... ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-19 17:09 Message: Logged In: YES user_id=357491 Some playing with gdb has turned up some clues. So LOAD_NAME is emitted by compiler_nameop(). How this partially works is that it gets the scope for an argument and then based on that emits the proper load, store, or delete. So why is xrange() coming out at the NAME scope? Well, turns out it is not being found in a particular scope by PyST_GetScope() and is thus returning 0 by default. This means that NAME becomes the scope for xrange(). So it looks like the symtable creation is screwing up and not making xrange() a global like it should. This might be a side-effect of the while closure thing above. Not sure, though. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-19 12:03 Message: Logged In: YES user_id=357491 Initially assigned to Nick since he did the genexp patch and in hopes he might know what is going on off the top of his head. Otherwise assign to me. I have a sneaking suspicion that the symtable code overall is slightly busted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 From noreply at sourceforge.net Wed Apr 20 04:31:51 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 04:31:55 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 21:03 Message generated for change (Comment added) made by lcaamano You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 22:31 Message: Logged In: YES user_id=279987 Clearly broken? Hardly. Daemonization code is not the only place where it's recommend and standard practice to close file descriptors. It's unreasonable to expect python programs to keep track of all the possible file descriptors the python library might cache to make sure it doesn't close them in all the daemonization routines ... btw, contrary to standard unix programming practices. Are there any other file descriptors we should know about? ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 18:27 Message: Logged In: YES user_id=81797 Perhaps the best way to resolve this would be for the standard library to provide code that either does the daemonize process, or at least does the closing of the sockets that may be done as part of the daemonize, that way it's clear what the "right" way is to do it. Thoughts? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-19 16:05 Message: Logged In: YES user_id=6380 I recommend to close this as invalid. The daemonization code is clearly broken. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-19 13:49 Message: Logged In: YES user_id=110477 Modifying the system os.py is not a good idea. A better work-around is to skip the /dev/urandom fd when you are closing all fds. This is the code we use: def close_fd(): # close all inherited file descriptors start_fd = 3 # Python 2.4.1 and later use /dev/urandom to seed the random module's RNG # a file descriptor is kept internally as os._urandomfd (created on demand # the first time os.urandom() is called), and should not be closed try: os.urandom(4) urandom_fd = getattr(os, '_urandomfd', None) except AttributeError: urandom_fd = None if '-close_fd' in sys.argv: start_fd = int(sys.argv[sys.argv.index('-close_fd') + 1]) for fd in range(start_fd, 256): if fd == urandom_fd: continue try: os.close(fd) except OSError: pass ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 12:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 03:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 23:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-05 21:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 20 04:49:05 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 04:49:09 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-06 01:03 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-20 02:49 Message: Logged In: YES user_id=81797 Conversely, I would say that it's unreasonable to expect other things not to break if you go through and close file descriptors that the standard library has opened. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-20 02:31 Message: Logged In: YES user_id=279987 Clearly broken? Hardly. Daemonization code is not the only place where it's recommend and standard practice to close file descriptors. It's unreasonable to expect python programs to keep track of all the possible file descriptors the python library might cache to make sure it doesn't close them in all the daemonization routines ... btw, contrary to standard unix programming practices. Are there any other file descriptors we should know about? ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:27 Message: Logged In: YES user_id=81797 Perhaps the best way to resolve this would be for the standard library to provide code that either does the daemonize process, or at least does the closing of the sockets that may be done as part of the daemonize, that way it's clear what the "right" way is to do it. Thoughts? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-19 20:05 Message: Logged In: YES user_id=6380 I recommend to close this as invalid. The daemonization code is clearly broken. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-19 17:49 Message: Logged In: YES user_id=110477 Modifying the system os.py is not a good idea. A better work-around is to skip the /dev/urandom fd when you are closing all fds. This is the code we use: def close_fd(): # close all inherited file descriptors start_fd = 3 # Python 2.4.1 and later use /dev/urandom to seed the random module's RNG # a file descriptor is kept internally as os._urandomfd (created on demand # the first time os.urandom() is called), and should not be closed try: os.urandom(4) urandom_fd = getattr(os, '_urandomfd', None) except AttributeError: urandom_fd = None if '-close_fd' in sys.argv: start_fd = int(sys.argv[sys.argv.index('-close_fd') + 1]) for fd in range(start_fd, 256): if fd == urandom_fd: continue try: os.close(fd) except OSError: pass ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 16:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 09:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 07:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 07:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 07:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 01:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 20 04:57:25 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 04:57:36 2005 Subject: [ python-Bugs-1120777 ] bug in unichr() documentation Message-ID: Bugs item #1120777, was opened at 2005-02-11 13:54 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1120777&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Marko Kreen (mkz) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: bug in unichr() documentation Initial Comment: http://www.python.org/doc/2.4/lib/built-in-funcs.html: > Return the Unicode string of one character whose Unicode > code is the integer i. > [...] > The argument must be in the range [0..65535], inclusive. unichr.__doc_ says: > Return a Unicode string of one character with ordinal i; 0 <= i <= 0x10ffff. Which is correct? ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-20 02:57 Message: Logged In: YES user_id=81797 Fred: The attached patch looks good to me. ---------------------------------------------------------------------- Comment By: Marko Kreen (mkz) Date: 2005-02-11 14:38 Message: Logged In: YES user_id=894541 Main problem for me was that the 65535 hints that unichr() may want UTF-16 values not Unicode. That was rather confusing. Ok, attached path clarifies unichr() range. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2005-02-11 14:03 Message: Logged In: YES user_id=38388 Whether unichr() handles the UCS2 or the UCS4 range depends on the configuration option you set at Python compile time. Perhaps we should extend the documentation to mention this difference ?! Doc patches are welcome :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1120777&group_id=5470 From noreply at sourceforge.net Wed Apr 20 05:31:36 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 05:31:39 2005 Subject: [ python-Bugs-1119439 ] Python Programming FAQ should be updated for Python 2.4 Message-ID: Bugs item #1119439, was opened at 2005-02-09 17:17 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 Category: Documentation Group: Python 2.4 >Status: Closed Resolution: None Priority: 5 Submitted By: Michael Hoffman (hoffmanm) Assigned to: Nobody/Anonymous (nobody) Summary: Python Programming FAQ should be updated for Python 2.4 Initial Comment: For example: * "1.4.3 How do I iterate over a sequence in reverse order?" should use reversed() * "1.4.4 How do you remove duplicates from a list?" should use set() * "1.5.2 I want to do a complicated sort: can you do a Schwartzian Transform in Python?" should use sort(key=) * section 1.6 should use new-style classes ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-20 03:31 Message: Logged In: YES user_id=81797 I've submitted this to the webmaster@python.org list for pursual. Michael Hoffman: If you can make a patch of the suggested changes, to help the process along. I'm considering it closed here, please let me know if you think is inappropriate? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 22:49 Message: Logged In: YES user_id=593130 I believe the FAQs are only maintained at www.python.org and not at SourceForge. If so, you should email your comments to webmaster@python.org and/or docs@python.org. I strongly urge you to work your comments up into a set of specific patches (in text form). In other words, give a specific suggested text to replace or augment the old one. Since there is no paid editor, this will make changes much easier and therefore more likely sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 From noreply at sourceforge.net Wed Apr 20 05:48:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 05:48:23 2005 Subject: [ python-Bugs-1117048 ] Dictionary Parsing Problem Message-ID: Bugs item #1117048, was opened at 2005-02-05 22:50 Message generated for change (Comment added) made by jafo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 2 Submitted By: WalterBrunswick (walterbrunswick) Assigned to: Nobody/Anonymous (nobody) Summary: Dictionary Parsing Problem Initial Comment: Python Version: Python 2.4 OS Version: Microsoft Windows XP [Version 5.1.2600] There may or may not be (according to some people on #python on irc.freenode.net) a fault, if you will, in the way that Python evaluates dictionaries (in this case, globals()). apparently printing of dicts are optimized not to produce the entire output string before printing it This may be an advantage when it comes to processing speed and memory management, but when it comes to catching errors, this could very well be a problem. (I am waltz.) ... you think it should print a dict with an error in it? ha! caught ya there! well, if there's an error, it should break. and it did. It shouldn't print AT ALL. why not? Because of the ERROR! ... I'm saying it should raise an abstract exception (something like DictError) BEFORE printing, as to allow the programmer to handle the error. Some people don't agree with my argument, but I'll let you (the core developers) decide what's best. I just thought I should mention this issue, to help improve Python to the max! ---------------------------------------------------------------------- >Comment By: Sean Reifschneider (jafo) Date: 2005-04-20 03:48 Message: Logged In: YES user_id=81797 Just to chime in, I am ok with the current behavior. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-17 04:30 Message: Logged In: YES user_id=593130 The Python interpreter does not print blank lines except as requested. While a starting linefeed would look nicer in this particular circumstance (error while outputting), I would not want it for two reasons: 1) the circumstance is extremely rare, should usually only result from a program bug, and cannot hardly be detected, so a starting linefeed would nearly always produce an extraneous blank line; 2) without the linefeed, it is completely clear that the error occured in mid-sentence, so to speak. Looking at the informative output, which I don't see as nasty, I see the following: * the error occurred while trying to get the repr of the object (a module) associated with the name 'const'. * in const, you defined a function with the special name __getattr__, which apparently gets treated as a method that overrides the default. (I was not aware you could do this for modules.) * because of the presence of __getattr__, or perhaps because of something else in const, repr() asked const, via __getattr__, whether it has a method __repr__. * your __getattr__ raised proprietary const.ConstError * since this is something other than the expected "return the (computed) attribute value or raise an AttributeError exception" (Ref Man 3.3.2 Customizing attribute access), this ended the globals() call and triggered a traceback. Without seeing the code for const, I wonder whether you really need a custom __setattr__ and __getattr__ (once fixed) or should get rid of them and use the default mechanism. Unless you see a real bug that I missed, please close this report. ---------------------------------------------------------------------- Comment By: Scott Baldwin (linkmastersab) Date: 2005-02-17 00:08 Message: Logged In: YES user_id=1135743 My opinion on the most feasible solution is to simple insert a linebreak before the traceback begins, so that the traceback stands out more. An exception for this kind of trivial bug is too expensive for any good that would come out of it. ---------------------------------------------------------------------- Comment By: Scott Baldwin (linkmastersab) Date: 2005-02-17 00:05 Message: Logged In: YES user_id=1135743 My opinion on the most feasible solution is to simple insert a linebreak before the traceback begins, so that the traceback stands out more. An exception for this kind of trivial bug is too expensive for any good that would come out of it. ---------------------------------------------------------------------- Comment By: WalterBrunswick (walterbrunswick) Date: 2005-02-16 23:05 Message: Logged In: YES user_id=1164416 The interpreter begins parsing the dictionary immediately, without buffering and checking for errors beforehand. In the case that an error occurs, such as a faulty key, the interpreter outputs a nasty-looking error - including the error message as part of the dictionary, as you can see from the log (ErrorLog2.log). There is no way that this can be prevented without locating the source of the problem. What I'm saying is that the interpreter should catch the error and raise an exception. But as you said, parsing data streams in chunks is standard; however, I can't see any way around this problem. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 20:07 Message: Logged In: YES user_id=593130 What do you mean by 'dict with an error in it'? I don't know that I can produce such a thing with Python. Or how globals could have an error. I suppose you could give a dict an unprintable key or value (by having __str__ raise an exception), but I would consider that a bug in the program and an error in the key/value and not the dict. I think the behavior on trying to print a collection with such nasty objects is best left undefined. Or do you have something else in mind? In any case, reading and writing data streams in chunks is standard practice. If a dict were large enough, its text representation would not even fit in RAM all at once. And, in any case ... Bug reports should give a specific example of the purported bug, complete with cut and paste of (minimal) code and output or error trace. They may need a reference to the relevant section of the manuals. If you don't have a verifiable example, please close this. Specific feature requests can be submitted as an RFE. Design discussions should be kept to discussion forums. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 From noreply at sourceforge.net Wed Apr 20 08:31:22 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 08:31:26 2005 Subject: [ python-Feature Requests-1185121 ] itertools.imerge: merge sequences Message-ID: Feature Requests item #1185121, was opened at 2005-04-18 12:11 Message generated for change (Settings changed) made by jneb You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 Category: Python Library Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Jurjen N.E. Bos (jneb) Assigned to: Raymond Hettinger (rhettinger) Summary: itertools.imerge: merge sequences Initial Comment: (For the itertools library, so Python 2.2 and up) This is a suggested addition to itertools, proposed name imerge. usage: imerge(seq0, seq1, ..., [key=]) result: imerge assumes the sequences are all in sorted order, and produces a iterator that returns pairs of the form (value, index), where value is a value of one of the sequences, and index is the index number of the given sequence. The output the imerge is in sorted order (taking into account the key function), so that identical values in the sequences will be produced from left to right. The code is surprisingly short, making use of the builtin heap module. (You may disagree with my style of argument handling; feel free to optimize it.) def imerge(*iterlist, **key): """Merge a sequence of sorted iterables. Returns pairs [value, index] where each value comes from iterlist[index], and the pairs are sorted if each of the iterators is sorted. Hint use groupby(imerge(...), operator.itemgetter(0)) to get the items one by one. """ if key.keys() not in ([], ["key"]): raise TypeError, "Excess keyword arguments for imerge" key = key.get("key", lambda x:x) from heapq import heapreplace, heappop #initialize the heap containing (inited, value, index, currentItem, iterator) #this automatically makes sure all iterators are initialized, then run, and finally emptied heap = [(False, None, index, None, iter(iterator)) for index, iterator in enumerate(iterlist)] while heap: inited, item, index, value, iterator = heap[0] if inited: yield value, index try: item = iterator.next() except StopIteration: heappop(heap) else: heapreplace(heap, (True, key(item), index, item, iterator)) If you find this little routine worth its size, please put it into itertools. - Jurjen ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 18:58 Message: Logged In: YES user_id=80475 For your specific application, it is better to use sorted(). When the underlying data consists of long runs of previously ordered data, sorted() will take advantage of that ordering and run in O(n) time. In contrast, using a heap will unnecessarily introduce O(n log n) behavior and not exploit the underlying data order. Recommend that you close this request. This discussion thus far confirms the original conclusion that imerge() use cases are dominated by sorted(chain(*iterlist)) which gives code that is shorter, faster, and easier to understand. ---------------------------------------------------------------------- Comment By: Jurjen N.E. Bos (jneb) Date: 2005-04-19 08:19 Message: Logged In: YES user_id=446428 Well, I was optimizing a piece of code with reasonbly long sorted lists (in memory, I agree) that were modified in all kinds of ways. I did not want the nlogn behaviour of sort, so I started writing a merge routine. I found out that the boundary cases of a merge implementation are a mess, until I disccovered the heap trick. Then I decided to clean it up and and put it up for a library routine. The fact that it uses iterators is obnly to make it more general, not specifically for the "lazy" properties. - Jurjen ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 22:43 Message: Logged In: YES user_id=80475 I had previously looked at an imerge() utility and found that it had only a single application (isomorphic to lazy mergesorting) and that the use cases were dominated by the in-memory alternative: sorted(chain(*iterlist)). Short of writing an external mergesort, what applications did you have in mind? What situations have you encountered where you have multiple sources of sorted data being generated on the fly (as opposed to already being in-memory), have needed one element at a time sequential access to a combined sort of that data, needed that combined sort only once, and could not afford to have the dataset in-memory? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185121&group_id=5470 From noreply at sourceforge.net Wed Apr 20 10:06:34 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 10:06:37 2005 Subject: [ python-Bugs-1119439 ] Python Programming FAQ should be updated for Python 2.4 Message-ID: Bugs item #1119439, was opened at 2005-02-09 17:17 Message generated for change (Comment added) made by hoffmanm You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 Category: Documentation Group: Python 2.4 Status: Closed Resolution: None Priority: 5 Submitted By: Michael Hoffman (hoffmanm) Assigned to: Nobody/Anonymous (nobody) Summary: Python Programming FAQ should be updated for Python 2.4 Initial Comment: For example: * "1.4.3 How do I iterate over a sequence in reverse order?" should use reversed() * "1.4.4 How do you remove duplicates from a list?" should use set() * "1.5.2 I want to do a complicated sort: can you do a Schwartzian Transform in Python?" should use sort(key=) * section 1.6 should use new-style classes ---------------------------------------------------------------------- >Comment By: Michael Hoffman (hoffmanm) Date: 2005-04-20 08:06 Message: Logged In: YES user_id=987664 That does not seem inappropriate. I unfortunately do not have time to do a real patch at this point. But this might be an excellent way for someone on comp.lang.python to help out. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-20 03:31 Message: Logged In: YES user_id=81797 I've submitted this to the webmaster@python.org list for pursual. Michael Hoffman: If you can make a patch of the suggested changes, to help the process along. I'm considering it closed here, please let me know if you think is inappropriate? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 22:49 Message: Logged In: YES user_id=593130 I believe the FAQs are only maintained at www.python.org and not at SourceForge. If so, you should email your comments to webmaster@python.org and/or docs@python.org. I strongly urge you to work your comments up into a set of specific patches (in text form). In other words, give a specific suggested text to replace or augment the old one. Since there is no paid editor, this will make changes much easier and therefore more likely sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 From noreply at sourceforge.net Wed Apr 20 13:36:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 13:36:46 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 09:56 Message generated for change (Comment added) made by yetanothermax You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- >Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:36 Message: Logged In: YES user_id=1218811 I've finally got access to a C compiler and it looks like the 'pure' Python changes for #754449 are in the clear (apologies to bcannon). The problem actually seems to relate to changes to threadmodule.c of the Python Modules directory. In versions greater than 2.3.4 it uses the PyGILState_XXX constructs t_bootstrap() function. These do not seem to be compatible with running sub interpreters as JEP (and probably mod_python) do. The python documentation (http://docs.python.org/api/threads.html) says "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python still supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_*() API is unsupported. " I've compiled a new versions of the python2X.dll for both 2.3.5 and 2.4.1 changing the use of these PyGIL_StateXXX constructs back to the 2.3.4 form and this seems to resolve the problem (I've attached the updated threadmodule.c files for both 2.3.5 and 2.4.1). ---------------------------------------------------------------------- Comment By: John Belmonte (jbelmonte) Date: 2005-04-16 22:27 Message: Logged In: YES user_id=282299 This problem also occurs on Linux. Note however that I've traced the source of the symptom to a different bug fix, namely [1010677] thread Module Breaks PyGILState_Ensure() (http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470). Specifically, the change from v2.56 to v2.56.8.1 of threadmodule.c. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-16 21:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-31 08:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Wed Apr 20 13:39:43 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 13:39:48 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 09:56 Message generated for change (Comment added) made by yetanothermax You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- >Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:39 Message: Logged In: YES user_id=1218811 Here's the updated threadmodule.c for 2.4.1 ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:36 Message: Logged In: YES user_id=1218811 I've finally got access to a C compiler and it looks like the 'pure' Python changes for #754449 are in the clear (apologies to bcannon). The problem actually seems to relate to changes to threadmodule.c of the Python Modules directory. In versions greater than 2.3.4 it uses the PyGILState_XXX constructs t_bootstrap() function. These do not seem to be compatible with running sub interpreters as JEP (and probably mod_python) do. The python documentation (http://docs.python.org/api/threads.html) says "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python still supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_*() API is unsupported. " I've compiled a new versions of the python2X.dll for both 2.3.5 and 2.4.1 changing the use of these PyGIL_StateXXX constructs back to the 2.3.4 form and this seems to resolve the problem (I've attached the updated threadmodule.c files for both 2.3.5 and 2.4.1). ---------------------------------------------------------------------- Comment By: John Belmonte (jbelmonte) Date: 2005-04-16 22:27 Message: Logged In: YES user_id=282299 This problem also occurs on Linux. Note however that I've traced the source of the symptom to a different bug fix, namely [1010677] thread Module Breaks PyGILState_Ensure() (http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470). Specifically, the change from v2.56 to v2.56.8.1 of threadmodule.c. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-16 21:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-31 08:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Wed Apr 20 13:51:11 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 13:51:16 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 09:56 Message generated for change (Comment added) made by yetanothermax You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- >Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:51 Message: Logged In: YES user_id=1218811 I didn't see jbelmote's comment, perhaps my updated threadmodule.c will reopen the #1010677 bug. ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:39 Message: Logged In: YES user_id=1218811 Here's the updated threadmodule.c for 2.4.1 ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:36 Message: Logged In: YES user_id=1218811 I've finally got access to a C compiler and it looks like the 'pure' Python changes for #754449 are in the clear (apologies to bcannon). The problem actually seems to relate to changes to threadmodule.c of the Python Modules directory. In versions greater than 2.3.4 it uses the PyGILState_XXX constructs t_bootstrap() function. These do not seem to be compatible with running sub interpreters as JEP (and probably mod_python) do. The python documentation (http://docs.python.org/api/threads.html) says "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python still supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_*() API is unsupported. " I've compiled a new versions of the python2X.dll for both 2.3.5 and 2.4.1 changing the use of these PyGIL_StateXXX constructs back to the 2.3.4 form and this seems to resolve the problem (I've attached the updated threadmodule.c files for both 2.3.5 and 2.4.1). ---------------------------------------------------------------------- Comment By: John Belmonte (jbelmonte) Date: 2005-04-16 22:27 Message: Logged In: YES user_id=282299 This problem also occurs on Linux. Note however that I've traced the source of the symptom to a different bug fix, namely [1010677] thread Module Breaks PyGILState_Ensure() (http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470). Specifically, the change from v2.56 to v2.56.8.1 of threadmodule.c. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-16 21:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-31 08:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Wed Apr 20 14:04:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 14:04:55 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 21:03 Message generated for change (Comment added) made by lcaamano You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-20 08:04 Message: Logged In: YES user_id=279987 I don't think so. One of the rules in libc, the standard C library, is that it cannot cache file descriptors for that same reason. This is not new. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:49 Message: Logged In: YES user_id=81797 Conversely, I would say that it's unreasonable to expect other things not to break if you go through and close file descriptors that the standard library has opened. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 22:31 Message: Logged In: YES user_id=279987 Clearly broken? Hardly. Daemonization code is not the only place where it's recommend and standard practice to close file descriptors. It's unreasonable to expect python programs to keep track of all the possible file descriptors the python library might cache to make sure it doesn't close them in all the daemonization routines ... btw, contrary to standard unix programming practices. Are there any other file descriptors we should know about? ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 18:27 Message: Logged In: YES user_id=81797 Perhaps the best way to resolve this would be for the standard library to provide code that either does the daemonize process, or at least does the closing of the sockets that may be done as part of the daemonize, that way it's clear what the "right" way is to do it. Thoughts? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-19 16:05 Message: Logged In: YES user_id=6380 I recommend to close this as invalid. The daemonization code is clearly broken. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-19 13:49 Message: Logged In: YES user_id=110477 Modifying the system os.py is not a good idea. A better work-around is to skip the /dev/urandom fd when you are closing all fds. This is the code we use: def close_fd(): # close all inherited file descriptors start_fd = 3 # Python 2.4.1 and later use /dev/urandom to seed the random module's RNG # a file descriptor is kept internally as os._urandomfd (created on demand # the first time os.urandom() is called), and should not be closed try: os.urandom(4) urandom_fd = getattr(os, '_urandomfd', None) except AttributeError: urandom_fd = None if '-close_fd' in sys.argv: start_fd = int(sys.argv[sys.argv.index('-close_fd') + 1]) for fd in range(start_fd, 256): if fd == urandom_fd: continue try: os.close(fd) except OSError: pass ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 12:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 03:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 23:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-05 21:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 20 14:59:50 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 14:59:55 2005 Subject: [ python-Bugs-1119439 ] Python Programming FAQ should be updated for Python 2.4 Message-ID: Bugs item #1119439, was opened at 2005-02-09 11:17 Message generated for change (Comment added) made by montanaro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 Category: Documentation Group: Python 2.4 >Status: Open Resolution: None Priority: 5 Submitted By: Michael Hoffman (hoffmanm) >Assigned to: Skip Montanaro (montanaro) Summary: Python Programming FAQ should be updated for Python 2.4 Initial Comment: For example: * "1.4.3 How do I iterate over a sequence in reverse order?" should use reversed() * "1.4.4 How do you remove duplicates from a list?" should use set() * "1.5.2 I want to do a complicated sort: can you do a Schwartzian Transform in Python?" should use sort(key=) * section 1.6 should use new-style classes ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 07:59 Message: Logged In: YES user_id=44345 Let's leave this open. It is a documentation bug/feature, so I think it was submitted to the correct place. Skip ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-04-20 03:06 Message: Logged In: YES user_id=987664 That does not seem inappropriate. I unfortunately do not have time to do a real patch at this point. But this might be an excellent way for someone on comp.lang.python to help out. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:31 Message: Logged In: YES user_id=81797 I've submitted this to the webmaster@python.org list for pursual. Michael Hoffman: If you can make a patch of the suggested changes, to help the process along. I'm considering it closed here, please let me know if you think is inappropriate? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 16:49 Message: Logged In: YES user_id=593130 I believe the FAQs are only maintained at www.python.org and not at SourceForge. If so, you should email your comments to webmaster@python.org and/or docs@python.org. I strongly urge you to work your comments up into a set of specific patches (in text form). In other words, give a specific suggested text to replace or augment the old one. Since there is no paid editor, this will make changes much easier and therefore more likely sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 From noreply at sourceforge.net Wed Apr 20 15:17:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 15:17:07 2005 Subject: [ python-Bugs-1177468 ] random.py/os.urandom robustness Message-ID: Bugs item #1177468, was opened at 2005-04-05 21:03 Message generated for change (Comment added) made by lcaamano You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: random.py/os.urandom robustness Initial Comment: Python 2.4.1 now uses os.urandom() to seed the random number generator. This is mostly an improvement, but can lead to subtle regression bugs. os.urandom() will open /dev/urandom on demand, e.g. when random.Random.seed() is called, and keep it alive as os._urandomfd. It is standard programming practice for a daemon process to close file descriptors it has inherited from its parent process, and if it closes the file descriptor corresponding to os._urandomfd, the os module is blissfully unaware and the next time os.urandom() is called, it will try to read from a closed file descriptor (or worse, a new one opened since), with unpredictable results. My recommendation would be to make os.urandom() open /dev/urandom each time and not keep a persistent file descripto. This will be slightly slower, but more robust. I am not sure how I feel about a standard library function steal a file descriptor slot forever, specially when os.urandom() is probably going to be called only once in the lifetime of a program, when the random module is seeded. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-20 09:17 Message: Logged In: YES user_id=279987 Here's a reference: http://tinyurl.com/b8mk3 The relevant post: ============================================ On 25 Feb 2001 10:48:22 GMT Casper H.S. Dik - Network Security Engineer wrote: | Solaris at various times used a cached /dev/zero fd both for mapping | thread stacks and even one for the runtime linker. | The runtime linker was mostly fine, but the thread library did have | problems with people closing fds. We since added MAP_ANON and no | longer require open("/dev/zero") . THe caaching of fds was gotten | rid of before that. | | There are valid reasons to close all fds; e.g., if you really don't | want to inherit and (you're a daemon and don't care). | | In most cases, though, the "close all" stuff performed by shells | and such at statup serves no purpose. (Other than causing more bugs ) So the dilemma is that closing fds can cause problems and leaving them open can cause problems, when a forked child does this. This seems to tell me that hiding fds in libraries and objects is a bad idea because processes need to know what is safe to close and/or what needs to be left open. ====================================== If the python library had some module or global list of opened file descriptors, then it would be OK to expect programs to keep those open across fork/exec. Something like: from os import opened_fds And then it would be no problem to skip those when closing fds. Otherwise, your nice daemon code that deals with _urandom_fd will break later on when somebody caches another fd somewhere else in the standard library. Also, the proposed os.daemonize() function that knows about its own fds would also work. Still, the most robust solution is not to cache open fds in the library or perhaps catch the EBADF exception and reopen. There are several solutions but closing this bug as invalid doesn't seem an appropriate one. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-20 08:04 Message: Logged In: YES user_id=279987 I don't think so. One of the rules in libc, the standard C library, is that it cannot cache file descriptors for that same reason. This is not new. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:49 Message: Logged In: YES user_id=81797 Conversely, I would say that it's unreasonable to expect other things not to break if you go through and close file descriptors that the standard library has opened. ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 22:31 Message: Logged In: YES user_id=279987 Clearly broken? Hardly. Daemonization code is not the only place where it's recommend and standard practice to close file descriptors. It's unreasonable to expect python programs to keep track of all the possible file descriptors the python library might cache to make sure it doesn't close them in all the daemonization routines ... btw, contrary to standard unix programming practices. Are there any other file descriptors we should know about? ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 18:27 Message: Logged In: YES user_id=81797 Perhaps the best way to resolve this would be for the standard library to provide code that either does the daemonize process, or at least does the closing of the sockets that may be done as part of the daemonize, that way it's clear what the "right" way is to do it. Thoughts? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2005-04-19 16:05 Message: Logged In: YES user_id=6380 I recommend to close this as invalid. The daemonization code is clearly broken. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-19 13:49 Message: Logged In: YES user_id=110477 Modifying the system os.py is not a good idea. A better work-around is to skip the /dev/urandom fd when you are closing all fds. This is the code we use: def close_fd(): # close all inherited file descriptors start_fd = 3 # Python 2.4.1 and later use /dev/urandom to seed the random module's RNG # a file descriptor is kept internally as os._urandomfd (created on demand # the first time os.urandom() is called), and should not be closed try: os.urandom(4) urandom_fd = getattr(os, '_urandomfd', None) except AttributeError: urandom_fd = None if '-close_fd' in sys.argv: start_fd = int(sys.argv[sys.argv.index('-close_fd') + 1]) for fd in range(start_fd, 256): if fd == urandom_fd: continue try: os.close(fd) except OSError: pass ---------------------------------------------------------------------- Comment By: Luis P Caamano (lcaamano) Date: 2005-04-19 12:53 Message: Logged In: YES user_id=279987 We're facing this problem. We're thinking of patching our os.py module to always open /dev/urandom on every call. Does anybody know if this would have any bad consequences other than the obvious system call overhead? BTW, here's the traceback we get. As you probably can guess, something called os.urandom before we closed all file descriptors in the daemonizing code and it then failed when os.urandom tried to use the cached fd. Traceback (most recent call last): File "/opt/race/share/sw/common/bin/dpmd", line 27, in ? dpmd().run() File "Linux/CommandLineApp.py", line 336, in run File "Linux/daemonbase.py", line 324, in main File "Linux/server.py", line 61, in addServices File "Linux/dpmd.py", line 293, in __init__ File "Linux/threadutils.py", line 44, in start File "Linux/xmlrpcd.py", line 165, in createThread File "Linux/threadutils.py", line 126, in __init__ File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 423, in NamedTemporaryFile dir = gettempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 262, in gettempdir tempdir = _get_default_tempdir() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 185, in _get_default_tempdir namer = _RandomNameSequence() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/t empfile.py", line 121, in __init__ self.rng = _Random() File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 96, in __init__ self.seed(x) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/r andom.py", line 110, in seed a = long(_hexlify(_urandom(16)), 16) File "/opt/race/share/sw/os/Linux_2.4_i686/python/lib/python2.4/ os.py", line 728, in urandom bytes += read(_urandomfd, n - len(bytes)) OSError : [Errno 9] Bad file descriptor ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 05:04 Message: Logged In: YES user_id=81797 The child is a copy of the parent. Therefore, if in the parent you open a few file descriptors, those are the ones you should close in the child. That is exactly what I've done in the past when I forked a child, and it has worked very well. I suspect Stevens would make an exception to his guideline in the event that closing a file descriptor results in library routine failures. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-06 03:27 Message: Logged In: YES user_id=110477 Unfortunately, catching exceptions is not sufficient - the file descriptor may have been reassigned. Fortunately in my case, to a socket which raised ENOSYS, but if it had been a normal file, this would have been much harder to trace because reading from it would cause weird errors for readers of the reassigned fd without triggering an exception in os.urandom() itself. As for not closing file descriptors you haven't opened yourself, if the process is the result of a vfork/exec (in my case Python processes started by a cluster manager, sort of like init), the child process has no clue what file descriptors, sockets or the like it has inherited from its parent process, and the safest course is to close them all. Indeed, that's what W. Richard Stevens recommends in "Advanced Programming for the UNIX environment". As far as I can tell, os.urandom() is used mostly to seed the RNG in random.py, and thus is not a high-drequency operation. It is going to be very hard to document this adequately for coders to defend against - in my case, the problem was being triggered by os.tempnam() from within Webware's PSP compiler. There are so many functions that depend on random (sometimes in non-obvious ways), you can't flag them all so their users know they should use urandomcleanup. One possible solution would be for os.py to offer a go_daemon() function that implements the fd closing, signal masking, process group and terminal disassociation required by true daemons. This function could take care of internal book-keeping like calling urandomcleanup. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-06 03:20 Message: Logged In: YES user_id=81797 Yeah, I was thinking the same thing. It doesn't address the consumed file handle, but it does address the "robustness" issue. It complicates the code, but should work. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-04-06 03:11 Message: Logged In: YES user_id=21627 To add robustness, it would be possible to catch read errors from _urandomfd, and try reopening it if it got somehow closed. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-05 23:11 Message: Logged In: YES user_id=81797 Just providing some feedback: I'm able to reproduce this. Importing random will cause this file descriptor to be called. Opening urandom on every call could lead to unacceptable syscall overhead for some. Perhaps there should be a "urandomcleanup" method that closes the file descriptor, and then random could get the bytes from urandom(), and clean up after itself? Personally, I only clean up the file descriptors I have allocated when I fork a new process. On the one hand I agree with you about sucking up a fd in the standard library, but on the other hand I'm thinking that you just shouldn't be closing file descriptors for stuff you'll be needing. That's my two cents on this bug. ---------------------------------------------------------------------- Comment By: Fazal Majid (majid) Date: 2005-04-05 21:06 Message: Logged In: YES user_id=110477 There are many modules that have a dependency on random, for instance os.tempnam(), and a program could well inadvertently use it before closing file descriptors. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1177468&group_id=5470 From noreply at sourceforge.net Wed Apr 20 20:02:28 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 20:02:31 2005 Subject: [ python-Bugs-1183585 ] try to open /dev/null as directory Message-ID: Bugs item #1183585, was opened at 2005-04-15 10:56 Message generated for change (Comment added) made by robang You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183585&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Roberto A. Foglietta (robang) Assigned to: Nobody/Anonymous (nobody) Summary: try to open /dev/null as directory Initial Comment: bash-2.05b# strace python 2>&1 | grep open | grep null open("/dev/null", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = -1 ENOTDIR (Not a directory) ---------------------------------------------------------------------- >Comment By: Roberto A. Foglietta (robang) Date: 2005-04-20 20:02 Message: Logged In: YES user_id=36141 I downloaded the python-2.4.1 sources and compiled under slack 10 and Mandrake 10.1 and in both case it tried to open /dev/null as a directory. Some debian users reported me the same behaviure. [roberto@wsraf roberto]$ uname -ar Linux wsraf.sad.it 2.6.8.1-24mdksmp #1 SMP Thu Jan 13 23:11:43 MST 2005 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz unknown GNU/Linux ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2005-04-16 09:54 Message: Logged In: YES user_id=849994 I don't quite understand what you're trying to tell us here. Are you just calling Python this way and observing the system calls? Then, what system do you use? What version? What kernel, etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1183585&group_id=5470 From noreply at sourceforge.net Wed Apr 20 20:55:36 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 20:55:41 2005 Subject: [ python-Bugs-1119439 ] Python Programming FAQ should be updated for Python 2.4 Message-ID: Bugs item #1119439, was opened at 2005-02-09 12:17 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hoffman (hoffmanm) Assigned to: Skip Montanaro (montanaro) Summary: Python Programming FAQ should be updated for Python 2.4 Initial Comment: For example: * "1.4.3 How do I iterate over a sequence in reverse order?" should use reversed() * "1.4.4 How do you remove duplicates from a list?" should use set() * "1.5.2 I want to do a complicated sort: can you do a Schwartzian Transform in Python?" should use sort(key=) * section 1.6 should use new-style classes ---------------------------------------------------------------------- >Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-20 14:55 Message: Logged In: YES user_id=593130 MH: by "a set of specific patches (in text form)" I meant an ascii text file with each item indicating a specific slice or location in the existing doc (possibly by quotation) and a suggested new text to insert. The site/doc maintainers have said that this is enough: if they agree with the new text, they will add the latex markup and produce the 'real patch'. Judging from your writing here, you seem quite capable of producing literate (ascii-text) sentences. SM: I currently disagree with this being reopened. 1) The items may need updating (like much library code) but this does not make them wrong (buggy). 2) The "Programmer's FAQ' is not part of the official doc defining the language and implementation nor, last I knew, is it part of the distribution. To me, it is one of hundred's of supplementary web pages, including the PEPs, some at python.org and some elsewhere. 3) the site gives its own address, not here, for submitting suggestions, like this, for improvememt. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 08:59 Message: Logged In: YES user_id=44345 Let's leave this open. It is a documentation bug/feature, so I think it was submitted to the correct place. Skip ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-04-20 04:06 Message: Logged In: YES user_id=987664 That does not seem inappropriate. I unfortunately do not have time to do a real patch at this point. But this might be an excellent way for someone on comp.lang.python to help out. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 23:31 Message: Logged In: YES user_id=81797 I've submitted this to the webmaster@python.org list for pursual. Michael Hoffman: If you can make a patch of the suggested changes, to help the process along. I'm considering it closed here, please let me know if you think is inappropriate? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 17:49 Message: Logged In: YES user_id=593130 I believe the FAQs are only maintained at www.python.org and not at SourceForge. If so, you should email your comments to webmaster@python.org and/or docs@python.org. I strongly urge you to work your comments up into a set of specific patches (in text form). In other words, give a specific suggested text to replace or augment the old one. Since there is no paid editor, this will make changes much easier and therefore more likely sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 From noreply at sourceforge.net Wed Apr 20 21:12:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 21:13:04 2005 Subject: [ python-Bugs-1119439 ] Python Programming FAQ should be updated for Python 2.4 Message-ID: Bugs item #1119439, was opened at 2005-02-09 12:17 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hoffman (hoffmanm) >Assigned to: A.M. Kuchling (akuchling) Summary: Python Programming FAQ should be updated for Python 2.4 Initial Comment: For example: * "1.4.3 How do I iterate over a sequence in reverse order?" should use reversed() * "1.4.4 How do you remove duplicates from a list?" should use set() * "1.5.2 I want to do a complicated sort: can you do a Schwartzian Transform in Python?" should use sort(key=) * section 1.6 should use new-style classes ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-20 14:12 Message: Logged In: YES user_id=80475 Andrew, do you have a preference as to whether this is kept on SF or done through pydotorg? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-20 13:55 Message: Logged In: YES user_id=593130 MH: by "a set of specific patches (in text form)" I meant an ascii text file with each item indicating a specific slice or location in the existing doc (possibly by quotation) and a suggested new text to insert. The site/doc maintainers have said that this is enough: if they agree with the new text, they will add the latex markup and produce the 'real patch'. Judging from your writing here, you seem quite capable of producing literate (ascii-text) sentences. SM: I currently disagree with this being reopened. 1) The items may need updating (like much library code) but this does not make them wrong (buggy). 2) The "Programmer's FAQ' is not part of the official doc defining the language and implementation nor, last I knew, is it part of the distribution. To me, it is one of hundred's of supplementary web pages, including the PEPs, some at python.org and some elsewhere. 3) the site gives its own address, not here, for submitting suggestions, like this, for improvememt. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 07:59 Message: Logged In: YES user_id=44345 Let's leave this open. It is a documentation bug/feature, so I think it was submitted to the correct place. Skip ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-04-20 03:06 Message: Logged In: YES user_id=987664 That does not seem inappropriate. I unfortunately do not have time to do a real patch at this point. But this might be an excellent way for someone on comp.lang.python to help out. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:31 Message: Logged In: YES user_id=81797 I've submitted this to the webmaster@python.org list for pursual. Michael Hoffman: If you can make a patch of the suggested changes, to help the process along. I'm considering it closed here, please let me know if you think is inappropriate? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 17:49 Message: Logged In: YES user_id=593130 I believe the FAQs are only maintained at www.python.org and not at SourceForge. If so, you should email your comments to webmaster@python.org and/or docs@python.org. I strongly urge you to work your comments up into a set of specific patches (in text form). In other words, give a specific suggested text to replace or augment the old one. Since there is no paid editor, this will make changes much easier and therefore more likely sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 From noreply at sourceforge.net Wed Apr 20 21:22:25 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 21:22:29 2005 Subject: [ python-Bugs-1119439 ] Python Programming FAQ should be updated for Python 2.4 Message-ID: Bugs item #1119439, was opened at 2005-02-09 11:17 Message generated for change (Comment added) made by montanaro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hoffman (hoffmanm) >Assigned to: Skip Montanaro (montanaro) Summary: Python Programming FAQ should be updated for Python 2.4 Initial Comment: For example: * "1.4.3 How do I iterate over a sequence in reverse order?" should use reversed() * "1.4.4 How do you remove duplicates from a list?" should use set() * "1.5.2 I want to do a complicated sort: can you do a Schwartzian Transform in Python?" should use sort(key=) * section 1.6 should use new-style classes ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 14:22 Message: Logged In: YES user_id=44345 In the absence of some other place to track this or a better category ("website"?) this seems as good a place as any to record the request. If we decide it's not a bug (advising people to use old solutions is not obviously a bug), then sure, close with "won't fix" or "invalid". Let's not simply lose the request though. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-20 14:12 Message: Logged In: YES user_id=80475 Andrew, do you have a preference as to whether this is kept on SF or done through pydotorg? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-20 13:55 Message: Logged In: YES user_id=593130 MH: by "a set of specific patches (in text form)" I meant an ascii text file with each item indicating a specific slice or location in the existing doc (possibly by quotation) and a suggested new text to insert. The site/doc maintainers have said that this is enough: if they agree with the new text, they will add the latex markup and produce the 'real patch'. Judging from your writing here, you seem quite capable of producing literate (ascii-text) sentences. SM: I currently disagree with this being reopened. 1) The items may need updating (like much library code) but this does not make them wrong (buggy). 2) The "Programmer's FAQ' is not part of the official doc defining the language and implementation nor, last I knew, is it part of the distribution. To me, it is one of hundred's of supplementary web pages, including the PEPs, some at python.org and some elsewhere. 3) the site gives its own address, not here, for submitting suggestions, like this, for improvememt. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 07:59 Message: Logged In: YES user_id=44345 Let's leave this open. It is a documentation bug/feature, so I think it was submitted to the correct place. Skip ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-04-20 03:06 Message: Logged In: YES user_id=987664 That does not seem inappropriate. I unfortunately do not have time to do a real patch at this point. But this might be an excellent way for someone on comp.lang.python to help out. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:31 Message: Logged In: YES user_id=81797 I've submitted this to the webmaster@python.org list for pursual. Michael Hoffman: If you can make a patch of the suggested changes, to help the process along. I'm considering it closed here, please let me know if you think is inappropriate? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 16:49 Message: Logged In: YES user_id=593130 I believe the FAQs are only maintained at www.python.org and not at SourceForge. If so, you should email your comments to webmaster@python.org and/or docs@python.org. I strongly urge you to work your comments up into a set of specific patches (in text form). In other words, give a specific suggested text to replace or augment the old one. Since there is no paid editor, this will make changes much easier and therefore more likely sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 From noreply at sourceforge.net Wed Apr 20 21:52:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 21:52:54 2005 Subject: [ python-Bugs-1186900 ] nntplib shouldn't raise generic EOFError Message-ID: Bugs item #1186900, was opened at 2005-04-20 12:52 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186900&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Matt Roper (mattrope) Assigned to: Nobody/Anonymous (nobody) Summary: nntplib shouldn't raise generic EOFError Initial Comment: Python's nntplib currently raises a generic EOFError if the connection is closed unexpectedly. This seems inconsistent with other Python libraries (smtplib, imaplib, etc.) and is unexpected behaviour. It seems that a new Exception class derived from the NNTPError (e.g., NNTPConnectionError) should be used instead. As it stands now, the only indication that EOFError can be raised is in the docstring for the internal getline() method. There is no mention in the documentation that higher level methods call getline() and can raise the EOFError to the application level. If no new exception class is added for this situation, it would be nice to have this behaviour noted in the documentation. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186900&group_id=5470 From noreply at sourceforge.net Wed Apr 20 21:53:38 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 21:53:41 2005 Subject: [ python-Bugs-1186900 ] nntplib shouldn't raise generic EOFError Message-ID: Bugs item #1186900, was opened at 2005-04-20 12:52 Message generated for change (Settings changed) made by mattrope You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186900&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None >Priority: 3 Submitted By: Matt Roper (mattrope) Assigned to: Nobody/Anonymous (nobody) Summary: nntplib shouldn't raise generic EOFError Initial Comment: Python's nntplib currently raises a generic EOFError if the connection is closed unexpectedly. This seems inconsistent with other Python libraries (smtplib, imaplib, etc.) and is unexpected behaviour. It seems that a new Exception class derived from the NNTPError (e.g., NNTPConnectionError) should be used instead. As it stands now, the only indication that EOFError can be raised is in the docstring for the internal getline() method. There is no mention in the documentation that higher level methods call getline() and can raise the EOFError to the application level. If no new exception class is added for this situation, it would be nice to have this behaviour noted in the documentation. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186900&group_id=5470 From noreply at sourceforge.net Wed Apr 20 21:57:27 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 21:57:30 2005 Subject: [ python-Bugs-1178148 ] cgitb.py report wrong line number Message-ID: Bugs item #1178148, was opened at 2005-04-07 00:04 Message generated for change (Comment added) made by barry-scott You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178148&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: Works For Me Priority: 5 Submitted By: Barry Alan Scott (barry-scott) Assigned to: Ka-Ping Yee (ping) Summary: cgitb.py report wrong line number Initial Comment: Given code like try: raise 'bug' except ValueError: pass # cgitb.py think 'bug' is here cgitb.py will report that the exception 'bug' is at the pass line. This is a time waster until you figure that this problem exists. ---------------------------------------------------------------------- >Comment By: Barry Alan Scott (barry-scott) Date: 2005-04-20 20:57 Message: Logged In: YES user_id=28665 I ran your example on Mac OS X 10.3.9 and its shipped Python 2.3 The attached exctest.html clear show the problem. Did this get fixed in 2.4? ---------------------------------------------------------------------- Comment By: Ka-Ping Yee (ping) Date: 2005-04-18 19:29 Message: Logged In: YES user_id=45338 Can't reproduce this problem. When i test the provided case, cgitb reports the error on the "raise" line as it should. See the attached file for my test case; cgitb highlights line 6 ("raise 'spam'"). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178148&group_id=5470 From noreply at sourceforge.net Wed Apr 20 22:04:07 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 20 22:04:13 2005 Subject: [ python-Bugs-1119439 ] Python Programming FAQ should be updated for Python 2.4 Message-ID: Bugs item #1119439, was opened at 2005-02-09 11:17 Message generated for change (Comment added) made by montanaro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hoffman (hoffmanm) >Assigned to: A.M. Kuchling (akuchling) Summary: Python Programming FAQ should be updated for Python 2.4 Initial Comment: For example: * "1.4.3 How do I iterate over a sequence in reverse order?" should use reversed() * "1.4.4 How do you remove duplicates from a list?" should use set() * "1.5.2 I want to do a complicated sort: can you do a Schwartzian Transform in Python?" should use sort(key=) * section 1.6 should use new-style classes ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 15:04 Message: Logged In: YES user_id=44345 (Last comment crossed paths with change to akuchling - restoring) ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 14:22 Message: Logged In: YES user_id=44345 In the absence of some other place to track this or a better category ("website"?) this seems as good a place as any to record the request. If we decide it's not a bug (advising people to use old solutions is not obviously a bug), then sure, close with "won't fix" or "invalid". Let's not simply lose the request though. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-20 14:12 Message: Logged In: YES user_id=80475 Andrew, do you have a preference as to whether this is kept on SF or done through pydotorg? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-20 13:55 Message: Logged In: YES user_id=593130 MH: by "a set of specific patches (in text form)" I meant an ascii text file with each item indicating a specific slice or location in the existing doc (possibly by quotation) and a suggested new text to insert. The site/doc maintainers have said that this is enough: if they agree with the new text, they will add the latex markup and produce the 'real patch'. Judging from your writing here, you seem quite capable of producing literate (ascii-text) sentences. SM: I currently disagree with this being reopened. 1) The items may need updating (like much library code) but this does not make them wrong (buggy). 2) The "Programmer's FAQ' is not part of the official doc defining the language and implementation nor, last I knew, is it part of the distribution. To me, it is one of hundred's of supplementary web pages, including the PEPs, some at python.org and some elsewhere. 3) the site gives its own address, not here, for submitting suggestions, like this, for improvememt. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2005-04-20 07:59 Message: Logged In: YES user_id=44345 Let's leave this open. It is a documentation bug/feature, so I think it was submitted to the correct place. Skip ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-04-20 03:06 Message: Logged In: YES user_id=987664 That does not seem inappropriate. I unfortunately do not have time to do a real patch at this point. But this might be an excellent way for someone on comp.lang.python to help out. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 22:31 Message: Logged In: YES user_id=81797 I've submitted this to the webmaster@python.org list for pursual. Michael Hoffman: If you can make a patch of the suggested changes, to help the process along. I'm considering it closed here, please let me know if you think is inappropriate? ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 16:49 Message: Logged In: YES user_id=593130 I believe the FAQs are only maintained at www.python.org and not at SourceForge. If so, you should email your comments to webmaster@python.org and/or docs@python.org. I strongly urge you to work your comments up into a set of specific patches (in text form). In other words, give a specific suggested text to replace or augment the old one. Since there is no paid editor, this will make changes much easier and therefore more likely sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1119439&group_id=5470 From noreply at sourceforge.net Thu Apr 21 00:13:50 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 00:13:52 2005 Subject: [ python-Bugs-1186195 ] [AST] genexps get scoping wrong Message-ID: Bugs item #1186195, was opened at 2005-04-20 05:02 Message generated for change (Comment added) made by ncoghlan You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nick Coghlan (ncoghlan) Summary: [AST] genexps get scoping wrong Initial Comment: test_genexps is failing because it is unable to find a global defined in a genexp that is returned. Here is the problem simplified: def f(n): return (i for i in xrange(n)) list(f(10)) Leads to ``SystemError: no locals when loading 'xrange'``. Comparing Python 2.4 bytecode: 1 0 LOAD_CONST 1 ( at 0x3931e0, file "", line 1>) 3 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (xrange) 9 LOAD_FAST 0 (n) 12 CALL_FUNCTION 1 15 GET_ITER 16 CALL_FUNCTION 1 19 RETURN_VALUE 20 LOAD_CONST 0 (None) 23 RETURN_VALUE to AST bytecode: 1 0 LOAD_CLOSURE 0 (n) 3 BUILD_TUPLE 1 6 LOAD_CONST 1 ( at 0x5212e8, file "", line 1>) 9 MAKE_CLOSURE 0 12 LOAD_NAME 0 (xrange) 15 LOAD_DEREF 0 (n) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 RETURN_VALUE 26 LOAD_CONST 0 (None) 29 RETURN_VALUE makes it obvious something is off (no peepholer; turned it off in my build of 2.4). Looks like extraneous work is being done in making a closure. Seems like it will still work, though. Plus the usage of LOAD_NAME is wrong in the AST; LOAD_NAME gets an object from the local namespace based on its name instead of offset. ---------------------------------------------------------------------- >Comment By: Nick Coghlan (ncoghlan) Date: 2005-04-21 08:13 Message: Logged In: YES user_id=1038590 The compiler stage does some fancy footwork so that the outermost iterable is evaluated immediately, and passed as an argument to the genexp block. The symbol table generation needs similar footwork, but it doesn't have it. That is, the symbol table for the outermost iterable is being created in the genexp scope, instead of in the containing function's scope, but the actual execution is happening in the function scope. Hilarity ensues. . . My original testing didn't pick this up because I was using (1, 2, 3) as the outermost iterable - no name lookups in the outermost iterable to be affected! If you haven't figured out a fix for this part of the problem before Sunday, I should be able to get to it then. The location of the enter_scope and exit_scope in the symbol table pass for genexps needs adjusting so that the outermost iterable is NOT part of the new scope. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-20 10:30 Message: Logged In: YES user_id=357491 OK, figured out why the closure thing is happening. 'n' is being detected as a free variable. This leads to PyCode_GetNumFree() to return a non-0 value. In compiler_make_closure() this automatically triggers the LOAD_CLOSURE/.../MAKE_CLOSURE chunk of bytecode instead of LOAD_CONST/MAKE_FUNCTION. So, how to make 'n' be detected as a local instead a free variable ... ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-20 10:09 Message: Logged In: YES user_id=357491 Some playing with gdb has turned up some clues. So LOAD_NAME is emitted by compiler_nameop(). How this partially works is that it gets the scope for an argument and then based on that emits the proper load, store, or delete. So why is xrange() coming out at the NAME scope? Well, turns out it is not being found in a particular scope by PyST_GetScope() and is thus returning 0 by default. This means that NAME becomes the scope for xrange(). So it looks like the symtable creation is screwing up and not making xrange() a global like it should. This might be a side-effect of the while closure thing above. Not sure, though. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-20 05:03 Message: Logged In: YES user_id=357491 Initially assigned to Nick since he did the genexp patch and in hopes he might know what is going on off the top of his head. Otherwise assign to me. I have a sneaking suspicion that the symtable code overall is slightly busted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 From noreply at sourceforge.net Thu Apr 21 05:11:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 05:11:09 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 19:56 Message generated for change (Comment added) made by mhammond You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- >Comment By: Mark Hammond (mhammond) Date: 2005-04-21 13:11 Message: Logged In: YES user_id=14198 Can anyone tell me specifically what Python C API function is involved here? ie, the "entry point" that these external threads use to call into Python. ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 21:51 Message: Logged In: YES user_id=1218811 I didn't see jbelmote's comment, perhaps my updated threadmodule.c will reopen the #1010677 bug. ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 21:39 Message: Logged In: YES user_id=1218811 Here's the updated threadmodule.c for 2.4.1 ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 21:36 Message: Logged In: YES user_id=1218811 I've finally got access to a C compiler and it looks like the 'pure' Python changes for #754449 are in the clear (apologies to bcannon). The problem actually seems to relate to changes to threadmodule.c of the Python Modules directory. In versions greater than 2.3.4 it uses the PyGILState_XXX constructs t_bootstrap() function. These do not seem to be compatible with running sub interpreters as JEP (and probably mod_python) do. The python documentation (http://docs.python.org/api/threads.html) says "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python still supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_*() API is unsupported. " I've compiled a new versions of the python2X.dll for both 2.3.5 and 2.4.1 changing the use of these PyGIL_StateXXX constructs back to the 2.3.4 form and this seems to resolve the problem (I've attached the updated threadmodule.c files for both 2.3.5 and 2.4.1). ---------------------------------------------------------------------- Comment By: John Belmonte (jbelmonte) Date: 2005-04-17 06:27 Message: Logged In: YES user_id=282299 This problem also occurs on Linux. Note however that I've traced the source of the symptom to a different bug fix, namely [1010677] thread Module Breaks PyGILState_Ensure() (http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470). Specifically, the change from v2.56 to v2.56.8.1 of threadmodule.c. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-17 05:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-31 16:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Thu Apr 21 07:51:09 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 07:51:20 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 00:56 Message generated for change (Comment added) made by mrjohnson0 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- Comment By: mrjohnson (mrjohnson0) Date: 2005-04-20 22:51 Message: Logged In: YES user_id=1044885 In interactive mode, Jep uses Py_NewInterpreter, Py_CompileString, and PyRun_String for its eval(). This file handles java -> python: http://204.156.146.230/cgi-bin/viewcvs.cgi/jep/pyembed.c?rev=1.29&content-type=text/vnd.viewcvs-markup You'd be looking for pyembed_eval(). ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2005-04-20 20:11 Message: Logged In: YES user_id=14198 Can anyone tell me specifically what Python C API function is involved here? ie, the "entry point" that these external threads use to call into Python. ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 04:51 Message: Logged In: YES user_id=1218811 I didn't see jbelmote's comment, perhaps my updated threadmodule.c will reopen the #1010677 bug. ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 04:39 Message: Logged In: YES user_id=1218811 Here's the updated threadmodule.c for 2.4.1 ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 04:36 Message: Logged In: YES user_id=1218811 I've finally got access to a C compiler and it looks like the 'pure' Python changes for #754449 are in the clear (apologies to bcannon). The problem actually seems to relate to changes to threadmodule.c of the Python Modules directory. In versions greater than 2.3.4 it uses the PyGILState_XXX constructs t_bootstrap() function. These do not seem to be compatible with running sub interpreters as JEP (and probably mod_python) do. The python documentation (http://docs.python.org/api/threads.html) says "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python still supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_*() API is unsupported. " I've compiled a new versions of the python2X.dll for both 2.3.5 and 2.4.1 changing the use of these PyGIL_StateXXX constructs back to the 2.3.4 form and this seems to resolve the problem (I've attached the updated threadmodule.c files for both 2.3.5 and 2.4.1). ---------------------------------------------------------------------- Comment By: John Belmonte (jbelmonte) Date: 2005-04-16 13:27 Message: Logged In: YES user_id=282299 This problem also occurs on Linux. Note however that I've traced the source of the symptom to a different bug fix, namely [1010677] thread Module Breaks PyGILState_Ensure() (http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470). Specifically, the change from v2.56 to v2.56.8.1 of threadmodule.c. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-16 12:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-30 22:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Thu Apr 21 08:13:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 08:13:51 2005 Subject: [ python-Bugs-1186195 ] [AST] genexps get scoping wrong Message-ID: Bugs item #1186195, was opened at 2005-04-19 12:02 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nick Coghlan (ncoghlan) Summary: [AST] genexps get scoping wrong Initial Comment: test_genexps is failing because it is unable to find a global defined in a genexp that is returned. Here is the problem simplified: def f(n): return (i for i in xrange(n)) list(f(10)) Leads to ``SystemError: no locals when loading 'xrange'``. Comparing Python 2.4 bytecode: 1 0 LOAD_CONST 1 ( at 0x3931e0, file "", line 1>) 3 MAKE_FUNCTION 0 6 LOAD_GLOBAL 0 (xrange) 9 LOAD_FAST 0 (n) 12 CALL_FUNCTION 1 15 GET_ITER 16 CALL_FUNCTION 1 19 RETURN_VALUE 20 LOAD_CONST 0 (None) 23 RETURN_VALUE to AST bytecode: 1 0 LOAD_CLOSURE 0 (n) 3 BUILD_TUPLE 1 6 LOAD_CONST 1 ( at 0x5212e8, file "", line 1>) 9 MAKE_CLOSURE 0 12 LOAD_NAME 0 (xrange) 15 LOAD_DEREF 0 (n) 18 CALL_FUNCTION 1 21 GET_ITER 22 CALL_FUNCTION 1 25 RETURN_VALUE 26 LOAD_CONST 0 (None) 29 RETURN_VALUE makes it obvious something is off (no peepholer; turned it off in my build of 2.4). Looks like extraneous work is being done in making a closure. Seems like it will still work, though. Plus the usage of LOAD_NAME is wrong in the AST; LOAD_NAME gets an object from the local namespace based on its name instead of offset. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-20 23:13 Message: Logged In: YES user_id=357491 OK, so basically what I figured out earlier today. I noticed that both genexps in 2.4 and the AST branch generated the same bytecode, so the only difference was how it was handling passing in arguments to the generator for its initial creation. I figured there was some jiggering in the compiler to note that the scoping in the genexp should not affect how variables were handled for its arguments. And as you said, the AST branch just doesn't take this into consideration as it should. I will see if I get to it, but I doubt I will (leaving town Friday for the weekend). I think this has justified fleshing out Python/compile.txt with a more thorough discussion of how the symbol table is generated. If you have any specific points you think should go in there, Nick (or anyone for that matter), please email them to me. Even explaining how the whole thing works would be a help. =) ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2005-04-20 15:13 Message: Logged In: YES user_id=1038590 The compiler stage does some fancy footwork so that the outermost iterable is evaluated immediately, and passed as an argument to the genexp block. The symbol table generation needs similar footwork, but it doesn't have it. That is, the symbol table for the outermost iterable is being created in the genexp scope, instead of in the containing function's scope, but the actual execution is happening in the function scope. Hilarity ensues. . . My original testing didn't pick this up because I was using (1, 2, 3) as the outermost iterable - no name lookups in the outermost iterable to be affected! If you haven't figured out a fix for this part of the problem before Sunday, I should be able to get to it then. The location of the enter_scope and exit_scope in the symbol table pass for genexps needs adjusting so that the outermost iterable is NOT part of the new scope. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-19 17:30 Message: Logged In: YES user_id=357491 OK, figured out why the closure thing is happening. 'n' is being detected as a free variable. This leads to PyCode_GetNumFree() to return a non-0 value. In compiler_make_closure() this automatically triggers the LOAD_CLOSURE/.../MAKE_CLOSURE chunk of bytecode instead of LOAD_CONST/MAKE_FUNCTION. So, how to make 'n' be detected as a local instead a free variable ... ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-19 17:09 Message: Logged In: YES user_id=357491 Some playing with gdb has turned up some clues. So LOAD_NAME is emitted by compiler_nameop(). How this partially works is that it gets the scope for an argument and then based on that emits the proper load, store, or delete. So why is xrange() coming out at the NAME scope? Well, turns out it is not being found in a particular scope by PyST_GetScope() and is thus returning 0 by default. This means that NAME becomes the scope for xrange(). So it looks like the symtable creation is screwing up and not making xrange() a global like it should. This might be a side-effect of the while closure thing above. Not sure, though. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-19 12:03 Message: Logged In: YES user_id=357491 Initially assigned to Nick since he did the genexp patch and in hopes he might know what is going on off the top of his head. Otherwise assign to me. I have a sneaking suspicion that the symtable code overall is slightly busted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186195&group_id=5470 From noreply at sourceforge.net Thu Apr 21 14:05:58 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 14:06:07 2005 Subject: [ python-Bugs-1163563 ] Sub threads execute in restricted mode Message-ID: Bugs item #1163563, was opened at 2005-03-15 09:56 Message generated for change (Comment added) made by yetanothermax You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 Category: Threads Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: anothermax (yetanothermax) Assigned to: Nobody/Anonymous (nobody) Summary: Sub threads execute in restricted mode Initial Comment: I'm using the JEP product which allows integration of Java with Python (see http://jepp.sourceforge.net) via starting a Python interpreter in the same process as the JVM. This integrates with python via the C interface, allowing the user to 'eval' python code (amongst other features). It seems that since Python 2.3.5 any threads (using the threading module) created via code that has been evaluated through the jep.eval() interface (i.e.Python C interface )are executed in restricted mode and cannot, for example, create files. Code that is just evaled (i.e not in a subthread) thread has no restrictions. The error reported is: IOError: file() constructor not accessible in restricted mode (see http://sourceforge.net/forum/forum.php? thread_id=1247793&forum_id=376782) - I've given a JEP specific example here. There seems to be a similar problem with mod_python (see http://www.modpython.org/pipermail/mod_python/2005- January/017129.html) Reading through the release notes for Python 2.3.5 I see: Bug #754449: threading.Thread will no longer mask exceptions raised during interpreter shutdown with another exception caused by attempting to output the initial exception. This fix also includes a backport of rev. 1.41 from HEAD. This might be the problem as it seems to involve the porting of 2.4 threading code back to the 2.3 tree. I've attached a Java file which shows the problem when using JEP. The error output is: Exception in thread Thread-1: Traceback (most recent call last): File "C:\Python24\Lib\threading.py", line 442, in __bootstrap File "", line 5, in run IOError: file() constructor not accessible in restricted mode 2.4.1c1 (#63, Mar 10 2005, 10:36:41) [MSC v.1310 32 bit (Intel)] Creating file from main thread... Done Creating file from sub thread... Done ---------------------------------------------------------------------- >Comment By: anothermax (yetanothermax) Date: 2005-04-21 14:05 Message: Logged In: YES user_id=1218811 I've attached a file (threadtest.c) that is adapted from the the way JEP uses Py_NewInterpreter. This runs correctly under 2.3.4 but fails under 2.3.5 and later Pythons. ---------------------------------------------------------------------- Comment By: mrjohnson (mrjohnson0) Date: 2005-04-21 07:51 Message: Logged In: YES user_id=1044885 In interactive mode, Jep uses Py_NewInterpreter, Py_CompileString, and PyRun_String for its eval(). This file handles java -> python: http://204.156.146.230/cgi-bin/viewcvs.cgi/jep/pyembed.c?rev=1.29&content-type=text/vnd.viewcvs-markup You'd be looking for pyembed_eval(). ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2005-04-21 05:11 Message: Logged In: YES user_id=14198 Can anyone tell me specifically what Python C API function is involved here? ie, the "entry point" that these external threads use to call into Python. ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:51 Message: Logged In: YES user_id=1218811 I didn't see jbelmote's comment, perhaps my updated threadmodule.c will reopen the #1010677 bug. ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:39 Message: Logged In: YES user_id=1218811 Here's the updated threadmodule.c for 2.4.1 ---------------------------------------------------------------------- Comment By: anothermax (yetanothermax) Date: 2005-04-20 13:36 Message: Logged In: YES user_id=1218811 I've finally got access to a C compiler and it looks like the 'pure' Python changes for #754449 are in the clear (apologies to bcannon). The problem actually seems to relate to changes to threadmodule.c of the Python Modules directory. In versions greater than 2.3.4 it uses the PyGILState_XXX constructs t_bootstrap() function. These do not seem to be compatible with running sub interpreters as JEP (and probably mod_python) do. The python documentation (http://docs.python.org/api/threads.html) says "Note that the PyGILState_*() functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python still supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_*() API is unsupported. " I've compiled a new versions of the python2X.dll for both 2.3.5 and 2.4.1 changing the use of these PyGIL_StateXXX constructs back to the 2.3.4 form and this seems to resolve the problem (I've attached the updated threadmodule.c files for both 2.3.5 and 2.4.1). ---------------------------------------------------------------------- Comment By: John Belmonte (jbelmonte) Date: 2005-04-16 22:27 Message: Logged In: YES user_id=282299 This problem also occurs on Linux. Note however that I've traced the source of the symptom to a different bug fix, namely [1010677] thread Module Breaks PyGILState_Ensure() (http://sourceforge.net/tracker/index.php?func=detail&aid=1010677&group_id=5470&atid=305470). Specifically, the change from v2.56 to v2.56.8.1 of threadmodule.c. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-16 21:21 Message: Logged In: YES user_id=357491 To answer the comment on bug #754449 and the email I got from someone, there is no way my fix for that bug would cause this. I only touched the 'threading' module, nothing else. That is in pure Python and thus has no direct way of modifying what does and does not execute in restricted mode. If I understand how restricted execution works that is all set at the C level. Plus the fix only deals with the teardown code; it in no way interfaces with how threads are executed. All changed code come after the thread and completed execution. The only thing it does ahead of time is store a reference to stdout. In other words it ain't my fault to the best of my knowledge. =) ---------------------------------------------------------------------- Comment By: Alan Davies (goatpunch) Date: 2005-03-31 08:24 Message: Logged In: YES user_id=967140 The same problem definately does occur with mod_python, I've found it to occur with Python 2.3.5 and 2.4 on Win32: http://www.modpython.org/pipermail/mod_python/2005- March/017802.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1163563&group_id=5470 From noreply at sourceforge.net Thu Apr 21 15:18:33 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 15:18:37 2005 Subject: [ python-Bugs-1184449 ] Read-only property attributes raise wrong exception Message-ID: Bugs item #1184449, was opened at 2005-04-16 19:16 Message generated for change (Comment added) made by bwarsaw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 Category: Python Interpreter Core Group: Python 2.5 >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Barry A. Warsaw (bwarsaw) Assigned to: Barry A. Warsaw (bwarsaw) Summary: Read-only property attributes raise wrong exception Initial Comment: In http://mail.python.org/pipermail/python-dev/2005-April/052681.html I describe how attributes defined as properties raise inconsistent errors depending on whether the property is defined in C or Python. By default in C, you'll get a TypeError while in Python you'll get an AttributeError. I think both should raise AttributeErrors. Here is a patch against Python 2.4 which should be ported to 2.5 (and possibly 2.3 if we care). ---------------------------------------------------------------------- >Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-21 09:18 Message: Logged In: YES user_id=12800 The change is in 2.5 and will /not/ be backported to 2.4. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2005-04-16 19:23 Message: Logged In: YES user_id=12800 Well, I was going to attach a patch, but it seems SF is borked and won't let me upload anything (yes the stupid checkbox was checked). The patch is simple though: change PyExc_TypeError to PyExc_AttributeError on lines 147 and 202 in Objects/descrobject.c (rev 2.38) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184449&group_id=5470 From noreply at sourceforge.net Thu Apr 21 16:05:18 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 16:05:21 2005 Subject: [ python-Bugs-1184380 ] example broken in section 1.12 of Extending & Embedding Message-ID: Bugs item #1184380, was opened at 2005-04-16 19:36 Message generated for change (Comment added) made by lbruno You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184380&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 1 Submitted By: bamoore (bamoore) Assigned to: Nobody/Anonymous (nobody) Summary: example broken in section 1.12 of Extending & Embedding Initial Comment: In section 1.12 of Extending and Embedding the Python Interpreter Release 2.4, the example (spammodule) does not compile on linux with distutils. I get the following: ~/sandbox/python/c_api_test$ python2.4 spam-setup.py build running build running build_ext building 'spam' extension creating build creating build/temp.linux-i686-2.4 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.4 -c spammodule.c -o build/temp.linux-i686-2.4/spammodule.o spammodule.c:7: error: conflicting types for `PySpam_System' spammodule.h:21: error: previous declaration of `PySpam_System' error: command 'gcc' failed with exit status 1 I think the solution is to change line 12 of the spammodule header file listing from: #define PySpam_System_PROTO (char *command) to: #define PySpam_System_PROTO (const char *command) as that seems to compile for me. (though not tested from another C extension). ---------------------------------------------------------------------- Comment By: Luis Bruno (lbruno) Date: 2005-04-21 14:05 Message: Logged In: YES user_id=1263795 AIUI, your solution is correct; a cursory glance through the PyArg_ParseTuple() documentation and the system() manual page leads me to believe we're expecting a const char *. I'll submit a patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1184380&group_id=5470 From noreply at sourceforge.net Thu Apr 21 16:58:15 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 16:58:18 2005 Subject: [ python-Bugs-1187437 ] TypeError message on bad iteration is misleading Message-ID: Bugs item #1187437, was opened at 2005-04-21 10:58 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1187437&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Roy Smith (roysmith) Assigned to: Nobody/Anonymous (nobody) Summary: TypeError message on bad iteration is misleading Initial Comment: >>> for i in 5: ... pass ... Traceback (most recent call last): File "", line 1, in ? TypeError: iteration over non-sequence I know this is kind of a trivial point, but the message "iteration of non-sequence" should really be something like "iteration of non-iterable object", since not all iterable things are sequences. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1187437&group_id=5470 From noreply at sourceforge.net Thu Apr 21 16:58:45 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 16:58:48 2005 Subject: [ python-Bugs-1187437 ] TypeError message on bad iteration is misleading Message-ID: Bugs item #1187437, was opened at 2005-04-21 10:58 Message generated for change (Settings changed) made by roysmith You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1187437&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None >Priority: 2 Submitted By: Roy Smith (roysmith) Assigned to: Nobody/Anonymous (nobody) Summary: TypeError message on bad iteration is misleading Initial Comment: >>> for i in 5: ... pass ... Traceback (most recent call last): File "", line 1, in ? TypeError: iteration over non-sequence I know this is kind of a trivial point, but the message "iteration of non-sequence" should really be something like "iteration of non-iterable object", since not all iterable things are sequences. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1187437&group_id=5470 From noreply at sourceforge.net Thu Apr 21 22:57:54 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 22:57:58 2005 Subject: [ python-Bugs-1178624 ] 2.4.1 breaks pyTTS Message-ID: Bugs item #1178624, was opened at 2005-04-07 17:25 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Dieter Deyke (deyke) Assigned to: Nobody/Anonymous (nobody) Summary: 2.4.1 breaks pyTTS Initial Comment: On my Windows XP PRO SP2 system, the combination of Python 2.4, pywin32-203.win32-py2.4, and pyTTS-3.0.win32-py2.4 did work. Updating to 2.4.1 broke pyTTS. Removing everythin, then installing 2.4.1, win32, and pyTTS resulted in the same failure. Removing everything again, and installing 2.4, win32, and pyTTS fixed it again. The problem is that with 2.4.1 pyTTS reported that "SAPI" was not supported. Looking deeper I saw that it got a syntax error reading a generated com interface file. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-21 22:57 Message: Logged In: YES user_id=89016 I think this is a duplicate of 1175396, so I'm closing the report. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 22:33 Message: Logged In: YES user_id=89016 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- Comment By: Dieter Deyke (deyke) Date: 2005-04-07 18:49 Message: Logged In: YES user_id=132928 I attached the generated interface file. Loading this file into python by itself does NOT generate an error, but going into the pyTTS dir, starting python, and typing "import sapi" shows: C:\Python24\Lib\site-packages\pyTTS>python Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sapi Traceback (most recent call last): File "", line 1, in ? File "C:\Python24\lib\site-packages\pyTTS\sapi.py", line 10, in ? sapi_mod = gencache.EnsureModule('{C866CA3A-32F7-11D2-9602-00C04F8EE628}', 0 , 5, 0) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 393, in EnsureModule module = GetModuleForTypelib(typelibCLSID, lcid, major, minor) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 258, in GetModuleForTypelib mod = _GetModule(modName) File "C:\Python24\lib\site-packages\win32com\client\gencache.py", line 629, in _GetModule mod = __import__(mod_name) File "C:\Python24\lib\site-packages\win32com\gen_py\C866CA3A-32F7-11D2-9602-00 C04F8EE628x0x5x0.py", line 1192 """Add""" ^ SyntaxError: invalid syntax >>> ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 17:56 Message: Logged In: YES user_id=89016 Can you post the generated inferface file? Does this file contain long lines? Does it use a PEP 263 encoding directive? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178624&group_id=5470 From noreply at sourceforge.net Thu Apr 21 23:50:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 23:50:20 2005 Subject: [ python-Bugs-1178484 ] Erroneous line number error in Py2.4.1 Message-ID: Bugs item #1178484, was opened at 2005-04-07 14:33 Message generated for change (Settings changed) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178484&group_id=5470 Category: Parser/Compiler Group: Python 2.4 Status: Open Resolution: Accepted Priority: 5 Submitted By: Timo Linna (tilinna) >Assigned to: Walter D?rwald (doerwalter) Summary: Erroneous line number error in Py2.4.1 Initial Comment: For some reason Python 2.3.5 reports the error in the following program correctly: File "C:\Temp\problem.py", line 7 SyntaxError: unknown decode error ..whereas Python 2.4.1 reports an invalid line number: File "C:\Temp\problem.py", line 2 SyntaxError: unknown decode error ----- problem.py starts ----- # -*- coding: ascii -*- """ Foo bar """ # ? is not allowed in ascii coding ----- problem.py ends ----- Without the encoding declaration both Python versions report the usual deprecation warning (just like they should be doing). My environment: Windows 2000 + SP3. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-07 16:28 Message: Logged In: YES user_id=89016 The reason for this is the new codec buffering code in 2.4: The codec might read and decode more data from the byte stream than is neccessary for decoding one line. I.e. when reading line n, the codec might decode bytes that belong to line n+1, n+2 etc. too. If there's a decoding error in this data, line n gets reported. I don't think there's a simple fix for this. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178484&group_id=5470 From noreply at sourceforge.net Thu Apr 21 23:55:32 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 21 23:55:35 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 17:14 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2005-04-21 23:55 Message: Logged In: YES user_id=89016 OK, I've checked in the following: Lib/codecs.py 1.44 Lib/test/test_codecs.py 1.23 Lib/codecs.py 1.35.2.7 Lib/test/test_codecs.py 1.15.2.5 with the following changes as suggested by glchapman: If a chunk read has a trailing "\r", read one more character even if the user has passed a size parameter. Remove the atcr logic. This should fix most of the problems. There are three open issues: 1) How do we handle the problem of a truncated line, if the data comes from the charbuffer instead of being read from the stream? 2) How do we handle error reporting? The buffering code might read more data than the current line. If this data has a decoding error the caller might report a wrong line number. (See bug #1178484) 3) Fixing the tokenizer. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-15 20:16 Message: Logged In: YES user_id=89016 The current readline() is implemented so that even if the complete line ending (i.e. '\r\n') can't be read within size bytes, at least something that might work as a line end (i.e. '\r') is available at the end of the string, so that the user knowns that the line is complete. The atcr members makes sure that the '\n' that is read next isn't misinterpreted as another line. Unfortunately the import mechanisn doesn't work that way: It demands a '\n' as line terminator and will continue reading if it only finds the '\r'. This means that the '\n' will be skipped and the import mechanisn treats those two lines as one. IMHO the best solution would be the read the extra character even if size is None, as glchapman suggested, so the above situation would really only happen if the last character from the stream is '\r'. I think the tokenizer should be able to survive this. What it didn't survive in 2.4 was that readline() tried to get it's hand on a complete line even if the length of the line was way bigger than the size passed in. If we remove the "size is None" restriction from the current code, then I think we should remove the atcr logic too, because it could only happen under exceptional circumstances and the old handling might be better for those users that don't recognize '\r' as a line end. But in any case the tokenizer should be fixed to be able to handle any line length returned from readline(). I'd really like to get a review by Martin v. L?wis of glchapman's patch #1101726. Of course the simplest solution would be: "If you want a complete line, don't pass a size parameter to readline()". I don't know if it would make sense to change the PEP263 machinery to support this. The other problem is if readline() returns data from the charbuffer. I don't really see how this can be fixed. We might call read() with a size parameter even if there are characters in the charbuffer, but what happens if the user calls readline(size=100000) first and then readline(size=10)? The first readline() call might put a very long string into the charbuffer and then the second call will return a unicode string whose length is in no way correlated to the size parameter. (This happens even even with the current code of course, so the message should be: Don't trust the size parameter, it only tells the underlying stream how many bytes to read (approximately), it's doesn't tell you *anything* about the amount of data returned.). This was different in 2.3.x, because the readline() method of the underlying stream was used, which handled this properly, but only worked if an encoded linefeed was recognizable as a normal 8bit '\n' linefeed by the underlying readline() (i.e. the UTF-16 encodings didn't work). ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-15 01:25 Message: Logged In: YES user_id=86307 I think the foo2.py from 1163244 is probably the same bug; at any rate, the reason for it is that a \r is at the beginning of the last line when read in by decoding_fgets. I have simpler test file which shows the bug which I'll email to Walter (you basically just have to get a \r as the last character in the block read by StreamReader, so that atcr will be true). The problem is caused by StreamReader.readline doing: if self.atcr and data.startswith(u"\n"): data = data[1:] since the tokenizer relies on '\n' as the line break character, but it will never see the '\n' removed by the above. FWIW (not much), I think the 2.4 StreamReader.readline actually made more sense than the current code, although a few changes would seem useful (see below). I don't think it is particularly useful to treat the size parameter as a fixed maximum number of bytes to read, since the number of bytes read has no fixed relationship to the number of decoded unicode characters (and also, in the case of the tokenizer, no fixed relationship to the number of bytes of encoded utf8). Also, given the current code, the size parameter is effectively ignored if there is a charbuffer: if you have 5 characters sitting in the charbuffer and use a size of 0x1FF, you only get back the 5 characters, even if they do not end in a linebreak. For the tokenizer, this means an unnecessary PyMem_RESIZE and an extra call to decoding_readline roughly every BUFSIZ bytes in the file (since the tokenizer assumes failure to fetch a complete line means its buffer is too small, whereas in fact it was caused by an incomplete line being stored in the StreamReader's charbuffer). As to changes from 2.4, if the unicode object were to add a findlinebreak method which returns the index of the first character for which Py_UNICODE_ISLINEBREAK is true, readline could use that instead of find("\n"). If it used such a method, readline would also need to explicitly handle a "\r\n" sequence, including a potential read(1) if a '\r' appears at the end of the data (in the case where size is not None). Of course, one problem with that idea is it requires a new method, which may not be allowed until 2.5, and the 2.4.1 behavior definitely needs to be fixed some way. (Interestingly, it looks to me like sre has everything necessary for searching for unicode linebreaks except syntax with which to express the idea in a pattern (maybe I'm missing something, but I can't find a way to get a compiled pattern to emit CATEGORY_UNI_LINEBREAK).) ---------------------------------------------------------------------- Comment By: Michal Rydlo (mmm) Date: 2005-04-15 00:04 Message: Logged In: YES user_id=65460 foo2.py from #1163244 fails to import. Not being expert in Python internals I hope it is due to this bug. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-11 23:42 Message: Logged In: YES user_id=89016 OK, I'm reopening to bug report. I didn't manage to install pythondoc. cElementTree complains about: No such file or directory: './pyconfig.h'. Can you provide a simple Python file that fails when imported? ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-10 00:47 Message: Logged In: YES user_id=86307 Sorry to comment on a closed report, but perhaps this fix should not be limited only to cases where size is None. Today, I ran into a spurious syntax error when trying to import pythondoc (from http://effbot.org/downloads/pythondoc-2.1b3-20050325.zip). It turned out a \r was ending up in what looked to the parser like the middle of a line, presumably because a \n was dropped. Anyway, I applied the referenced patch to 2.4.1, except I left out the "size is None" condition, since I knew the tokenizer passes in a size param. With that change pythondoc import successfully. (Also, I just ran the test suite and nothing broke.) Since the size parameter is already documented as being passed to StreamReader.read (in codecs.py -- the HTML documentation needs to be updated), and since StreamReader.read says size is an approximate maximum, perhaps it's OK to read one extra byte. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-05 00:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Fri Apr 22 04:59:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 22 04:59:50 2005 Subject: [ python-Bugs-1117048 ] Dictionary Parsing Problem Message-ID: Bugs item #1117048, was opened at 2005-02-05 17:50 Message generated for change (Comment added) made by walterbrunswick You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 2 Submitted By: WalterBrunswick (walterbrunswick) Assigned to: Nobody/Anonymous (nobody) Summary: Dictionary Parsing Problem Initial Comment: Python Version: Python 2.4 OS Version: Microsoft Windows XP [Version 5.1.2600] There may or may not be (according to some people on #python on irc.freenode.net) a fault, if you will, in the way that Python evaluates dictionaries (in this case, globals()). apparently printing of dicts are optimized not to produce the entire output string before printing it This may be an advantage when it comes to processing speed and memory management, but when it comes to catching errors, this could very well be a problem. (I am waltz.) ... you think it should print a dict with an error in it? ha! caught ya there! well, if there's an error, it should break. and it did. It shouldn't print AT ALL. why not? Because of the ERROR! ... I'm saying it should raise an abstract exception (something like DictError) BEFORE printing, as to allow the programmer to handle the error. Some people don't agree with my argument, but I'll let you (the core developers) decide what's best. I just thought I should mention this issue, to help improve Python to the max! ---------------------------------------------------------------------- >Comment By: WalterBrunswick (walterbrunswick) Date: 2005-04-21 22:59 Message: Logged In: YES user_id=1164416 As much as I'd like to take credit for writing the constants module, credit goes to Alex Martelli who submitted the script to the ActiveState Programmer Network (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/652 07); however, I did append the __getattr__ attribute, as well as __delattr__ -- that is where the errors occurred. But again, lest we forget, the issue is the dictionary parsing problem, which I have clearly marked. But as it has been mentioned, data is streamed in blocks rather than buffered, as to save memory. I'm closing this report. Sorry for any inconvenience. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 23:48 Message: Logged In: YES user_id=81797 Just to chime in, I am ok with the current behavior. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 23:30 Message: Logged In: YES user_id=593130 The Python interpreter does not print blank lines except as requested. While a starting linefeed would look nicer in this particular circumstance (error while outputting), I would not want it for two reasons: 1) the circumstance is extremely rare, should usually only result from a program bug, and cannot hardly be detected, so a starting linefeed would nearly always produce an extraneous blank line; 2) without the linefeed, it is completely clear that the error occured in mid-sentence, so to speak. Looking at the informative output, which I don't see as nasty, I see the following: * the error occurred while trying to get the repr of the object (a module) associated with the name 'const'. * in const, you defined a function with the special name __getattr__, which apparently gets treated as a method that overrides the default. (I was not aware you could do this for modules.) * because of the presence of __getattr__, or perhaps because of something else in const, repr() asked const, via __getattr__, whether it has a method __repr__. * your __getattr__ raised proprietary const.ConstError * since this is something other than the expected "return the (computed) attribute value or raise an AttributeError exception" (Ref Man 3.3.2 Customizing attribute access), this ended the globals() call and triggered a traceback. Without seeing the code for const, I wonder whether you really need a custom __setattr__ and __getattr__ (once fixed) or should get rid of them and use the default mechanism. Unless you see a real bug that I missed, please close this report. ---------------------------------------------------------------------- Comment By: Scott Baldwin (linkmastersab) Date: 2005-02-16 19:08 Message: Logged In: YES user_id=1135743 My opinion on the most feasible solution is to simple insert a linebreak before the traceback begins, so that the traceback stands out more. An exception for this kind of trivial bug is too expensive for any good that would come out of it. ---------------------------------------------------------------------- Comment By: Scott Baldwin (linkmastersab) Date: 2005-02-16 19:05 Message: Logged In: YES user_id=1135743 My opinion on the most feasible solution is to simple insert a linebreak before the traceback begins, so that the traceback stands out more. An exception for this kind of trivial bug is too expensive for any good that would come out of it. ---------------------------------------------------------------------- Comment By: WalterBrunswick (walterbrunswick) Date: 2005-02-16 18:05 Message: Logged In: YES user_id=1164416 The interpreter begins parsing the dictionary immediately, without buffering and checking for errors beforehand. In the case that an error occurs, such as a faulty key, the interpreter outputs a nasty-looking error - including the error message as part of the dictionary, as you can see from the log (ErrorLog2.log). There is no way that this can be prevented without locating the source of the problem. What I'm saying is that the interpreter should catch the error and raise an exception. But as you said, parsing data streams in chunks is standard; however, I can't see any way around this problem. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 15:07 Message: Logged In: YES user_id=593130 What do you mean by 'dict with an error in it'? I don't know that I can produce such a thing with Python. Or how globals could have an error. I suppose you could give a dict an unprintable key or value (by having __str__ raise an exception), but I would consider that a bug in the program and an error in the key/value and not the dict. I think the behavior on trying to print a collection with such nasty objects is best left undefined. Or do you have something else in mind? In any case, reading and writing data streams in chunks is standard practice. If a dict were large enough, its text representation would not even fit in RAM all at once. And, in any case ... Bug reports should give a specific example of the purported bug, complete with cut and paste of (minimal) code and output or error trace. They may need a reference to the relevant section of the manuals. If you don't have a verifiable example, please close this. Specific feature requests can be submitted as an RFE. Design discussions should be kept to discussion forums. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 From noreply at sourceforge.net Fri Apr 22 18:02:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 22 18:02:49 2005 Subject: [ python-Bugs-1188175 ] Pickle with HIGHEST_PROTOCOL "ord() expected..." Message-ID: Bugs item #1188175, was opened at 2005-04-22 16:02 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188175&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Selber (drhok) Assigned to: Nobody/Anonymous (nobody) Summary: Pickle with HIGHEST_PROTOCOL "ord() expected..." Initial Comment: A Python script that I wrote contains a tree-like data structure that cannot be unpickled when pickled with protocol==-1 (or 2). I attached the smallest example I could find. If run with pickle, the result is: Traceback (most recent call last): File "D:\src\misc\CDROM\python\pickletest.py", line 68, in ? tree=load( f ) File "C:\Python23\lib\pickle.py", line 1390, in load return Unpickler(file).load() File "C:\Python23\lib\pickle.py", line 872, in load dispatch[key](self) File "C:\Python23\lib\pickle.py", line 1189, in load_binput i = ord(self.read(1)) TypeError: ord() expected a character, but string of length 0 found But with cPickle, the result is: Traceback (most recent call last): File "D:\src\misc\CDROM\python\pickletest.py", line 68, in ? tree=load( f ) EOFError Small changes in the datastructure lead to different results: try changing the line if level<4: to if level<1: Now cPickle works, but not pickle. This happens with Python 2.3.3 and 2.3.5 on Windows XP. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188175&group_id=5470 From noreply at sourceforge.net Fri Apr 22 19:01:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 22 19:01:23 2005 Subject: [ python-Bugs-1188175 ] Pickle with HIGHEST_PROTOCOL "ord() expected..." Message-ID: Bugs item #1188175, was opened at 2005-04-22 12:02 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188175&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Selber (drhok) Assigned to: Nobody/Anonymous (nobody) Summary: Pickle with HIGHEST_PROTOCOL "ord() expected..." Initial Comment: A Python script that I wrote contains a tree-like data structure that cannot be unpickled when pickled with protocol==-1 (or 2). I attached the smallest example I could find. If run with pickle, the result is: Traceback (most recent call last): File "D:\src\misc\CDROM\python\pickletest.py", line 68, in ? tree=load( f ) File "C:\Python23\lib\pickle.py", line 1390, in load return Unpickler(file).load() File "C:\Python23\lib\pickle.py", line 872, in load dispatch[key](self) File "C:\Python23\lib\pickle.py", line 1189, in load_binput i = ord(self.read(1)) TypeError: ord() expected a character, but string of length 0 found But with cPickle, the result is: Traceback (most recent call last): File "D:\src\misc\CDROM\python\pickletest.py", line 68, in ? tree=load( f ) EOFError Small changes in the datastructure lead to different results: try changing the line if level<4: to if level<1: Now cPickle works, but not pickle. This happens with Python 2.3.3 and 2.3.5 on Windows XP. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-22 13:01 Message: Logged In: YES user_id=31435 How did you open the file? You must open files holding pickle data in binary mode on Windows (and should open them in binary mode on all platforms). IOW, you must do f = open(whatever, 'wb') when opening a file to which you write a pickle, and must use f = open(whatever, 'rb') when opening a file from which you want to read a pickle. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188175&group_id=5470 From noreply at sourceforge.net Fri Apr 22 19:09:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 22 19:09:55 2005 Subject: [ python-Bugs-1117048 ] Dictionary Parsing Problem Message-ID: Bugs item #1117048, was opened at 2005-02-05 17:50 Message generated for change (Settings changed) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Invalid Priority: 2 Submitted By: WalterBrunswick (walterbrunswick) Assigned to: Nobody/Anonymous (nobody) Summary: Dictionary Parsing Problem Initial Comment: Python Version: Python 2.4 OS Version: Microsoft Windows XP [Version 5.1.2600] There may or may not be (according to some people on #python on irc.freenode.net) a fault, if you will, in the way that Python evaluates dictionaries (in this case, globals()). apparently printing of dicts are optimized not to produce the entire output string before printing it This may be an advantage when it comes to processing speed and memory management, but when it comes to catching errors, this could very well be a problem. (I am waltz.) ... you think it should print a dict with an error in it? ha! caught ya there! well, if there's an error, it should break. and it did. It shouldn't print AT ALL. why not? Because of the ERROR! ... I'm saying it should raise an abstract exception (something like DictError) BEFORE printing, as to allow the programmer to handle the error. Some people don't agree with my argument, but I'll let you (the core developers) decide what's best. I just thought I should mention this issue, to help improve Python to the max! ---------------------------------------------------------------------- Comment By: WalterBrunswick (walterbrunswick) Date: 2005-04-21 22:59 Message: Logged In: YES user_id=1164416 As much as I'd like to take credit for writing the constants module, credit goes to Alex Martelli who submitted the script to the ActiveState Programmer Network (http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/652 07); however, I did append the __getattr__ attribute, as well as __delattr__ -- that is where the errors occurred. But again, lest we forget, the issue is the dictionary parsing problem, which I have clearly marked. But as it has been mentioned, data is streamed in blocks rather than buffered, as to save memory. I'm closing this report. Sorry for any inconvenience. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2005-04-19 23:48 Message: Logged In: YES user_id=81797 Just to chime in, I am ok with the current behavior. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 23:30 Message: Logged In: YES user_id=593130 The Python interpreter does not print blank lines except as requested. While a starting linefeed would look nicer in this particular circumstance (error while outputting), I would not want it for two reasons: 1) the circumstance is extremely rare, should usually only result from a program bug, and cannot hardly be detected, so a starting linefeed would nearly always produce an extraneous blank line; 2) without the linefeed, it is completely clear that the error occured in mid-sentence, so to speak. Looking at the informative output, which I don't see as nasty, I see the following: * the error occurred while trying to get the repr of the object (a module) associated with the name 'const'. * in const, you defined a function with the special name __getattr__, which apparently gets treated as a method that overrides the default. (I was not aware you could do this for modules.) * because of the presence of __getattr__, or perhaps because of something else in const, repr() asked const, via __getattr__, whether it has a method __repr__. * your __getattr__ raised proprietary const.ConstError * since this is something other than the expected "return the (computed) attribute value or raise an AttributeError exception" (Ref Man 3.3.2 Customizing attribute access), this ended the globals() call and triggered a traceback. Without seeing the code for const, I wonder whether you really need a custom __setattr__ and __getattr__ (once fixed) or should get rid of them and use the default mechanism. Unless you see a real bug that I missed, please close this report. ---------------------------------------------------------------------- Comment By: Scott Baldwin (linkmastersab) Date: 2005-02-16 19:08 Message: Logged In: YES user_id=1135743 My opinion on the most feasible solution is to simple insert a linebreak before the traceback begins, so that the traceback stands out more. An exception for this kind of trivial bug is too expensive for any good that would come out of it. ---------------------------------------------------------------------- Comment By: Scott Baldwin (linkmastersab) Date: 2005-02-16 19:05 Message: Logged In: YES user_id=1135743 My opinion on the most feasible solution is to simple insert a linebreak before the traceback begins, so that the traceback stands out more. An exception for this kind of trivial bug is too expensive for any good that would come out of it. ---------------------------------------------------------------------- Comment By: WalterBrunswick (walterbrunswick) Date: 2005-02-16 18:05 Message: Logged In: YES user_id=1164416 The interpreter begins parsing the dictionary immediately, without buffering and checking for errors beforehand. In the case that an error occurs, such as a faulty key, the interpreter outputs a nasty-looking error - including the error message as part of the dictionary, as you can see from the log (ErrorLog2.log). There is no way that this can be prevented without locating the source of the problem. What I'm saying is that the interpreter should catch the error and raise an exception. But as you said, parsing data streams in chunks is standard; however, I can't see any way around this problem. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2005-02-16 15:07 Message: Logged In: YES user_id=593130 What do you mean by 'dict with an error in it'? I don't know that I can produce such a thing with Python. Or how globals could have an error. I suppose you could give a dict an unprintable key or value (by having __str__ raise an exception), but I would consider that a bug in the program and an error in the key/value and not the dict. I think the behavior on trying to print a collection with such nasty objects is best left undefined. Or do you have something else in mind? In any case, reading and writing data streams in chunks is standard practice. If a dict were large enough, its text representation would not even fit in RAM all at once. And, in any case ... Bug reports should give a specific example of the purported bug, complete with cut and paste of (minimal) code and output or error trace. They may need a reference to the relevant section of the manuals. If you don't have a verifiable example, please close this. Specific feature requests can be submitted as an RFE. Design discussions should be kept to discussion forums. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1117048&group_id=5470 From noreply at sourceforge.net Fri Apr 22 20:21:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 22 20:21:22 2005 Subject: [ python-Bugs-1188231 ] Rebuilding from source on RH9 fails (_tkinter.so missing) Message-ID: Bugs item #1188231, was opened at 2005-04-22 11:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188231&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Marty Heyman (mheyman) Assigned to: Nobody/Anonymous (nobody) Summary: Rebuilding from source on RH9 fails (_tkinter.so missing) Initial Comment: On a Red Hat 9 system, I downloaded the python2.4-2.4. 1-1pydotorg.src.rpm and, following the web page ran "rpm --rebuild . ..". It went a long for a good long while with no apparent errors and then said: --- RPM build errors: File not found by glob: /var/tmp/python2.4-2.4. 1-root/usr/lib/python2.4/lib-dynload/_tkinter.so* --- I looked in the directory and there is, in fact, no _tkinter.so file(s) there. -- Marty Heyman ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188231&group_id=5470 From noreply at sourceforge.net Fri Apr 22 20:42:08 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 22 20:42:10 2005 Subject: [ python-Bugs-1188231 ] Rebuilding from source on RH9 fails (_tkinter.so missing) Message-ID: Bugs item #1188231, was opened at 2005-04-22 11:21 Message generated for change (Comment added) made by mheyman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188231&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Marty Heyman (mheyman) Assigned to: Nobody/Anonymous (nobody) Summary: Rebuilding from source on RH9 fails (_tkinter.so missing) Initial Comment: On a Red Hat 9 system, I downloaded the python2.4-2.4. 1-1pydotorg.src.rpm and, following the web page ran "rpm --rebuild . ..". It went a long for a good long while with no apparent errors and then said: --- RPM build errors: File not found by glob: /var/tmp/python2.4-2.4. 1-root/usr/lib/python2.4/lib-dynload/_tkinter.so* --- I looked in the directory and there is, in fact, no _tkinter.so file(s) there. -- Marty Heyman ---------------------------------------------------------------------- >Comment By: Marty Heyman (mheyman) Date: 2005-04-22 11:42 Message: Logged In: YES user_id=421967 APOLOGIES: ADDITIONAL INFO FOLLOWS ---Snip from rebuild output follows In file included from /usr/src/redhat/BUILD/Python-2.4.1/Modules/_tkinter.c:67: /usr/include/tk.h:83:29: X11/Xlib.h: No such file or directory In file included from /usr/src/redhat/BUILD/Python-2.4.1/Modules/_tkinter.c:67: /usr/include/tk.h:581: parse error before "Bool" /usr/include/tk.h:583: parse error before "event" /usr/include/tk.h:584: parse error before "root" /usr/include/tk.h:585: parse error before "subwindow" /usr/include/tk.h:586: parse error before "time" /usr/include/tk.h:586: `time' redeclared as different kind of symbol /usr/include/time.h:184: previous declaration of `time' /usr/include/tk.h:591: parse error before "same_screen" --- snip ends many more "parse error lines occurred after this. I doubt they're interesting . A bit later, another group of failures begins --Snip starts In file included from /usr/include/tk.h:1361, from /usr/src/redhat/BUILD/Python-2.4.1/Modules/_tkinter.c:67: /usr/include/tkDecls.h:37: parse error before '*' token /usr/include/tkDecls.h:39: parse error before "Tk_3DBorderGC" /usr/include/tkDecls.h:45: parse error before "Drawable" /usr/include/tkDecls.h:50: parse error before "Drawable" /usr/include/tkDecls.h:58: parse error before "XEvent" --Snip ends Again, there are many more similar messages following those and then: --Snip starts /usr/include/tkDecls.h:843: `GC' declared as function returning a function ... [parse errors] /usr/include/tkDecls.h:906: `Font' declared as function returning a function --Snip ends There are many such embedded in that group. Then the messages stop followed by a line that says: "running build_scripts" ... and things proceed as if all was OK for hundreds more lines of output before the failure copied into the original report. I captured the complete log and can upload it if needed on request. I'd need to trim it and it doesn't look all that interesting otherwise but then, what do I know :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188231&group_id=5470 From noreply at sourceforge.net Fri Apr 22 22:54:38 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 22 22:54:40 2005 Subject: [ python-Bugs-1188175 ] Pickle with HIGHEST_PROTOCOL "ord() expected..." Message-ID: Bugs item #1188175, was opened at 2005-04-22 16:02 Message generated for change (Comment added) made by drhok You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188175&group_id=5470 Category: Python Library Group: Python 2.3 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Heiko Selber (drhok) Assigned to: Nobody/Anonymous (nobody) Summary: Pickle with HIGHEST_PROTOCOL "ord() expected..." Initial Comment: A Python script that I wrote contains a tree-like data structure that cannot be unpickled when pickled with protocol==-1 (or 2). I attached the smallest example I could find. If run with pickle, the result is: Traceback (most recent call last): File "D:\src\misc\CDROM\python\pickletest.py", line 68, in ? tree=load( f ) File "C:\Python23\lib\pickle.py", line 1390, in load return Unpickler(file).load() File "C:\Python23\lib\pickle.py", line 872, in load dispatch[key](self) File "C:\Python23\lib\pickle.py", line 1189, in load_binput i = ord(self.read(1)) TypeError: ord() expected a character, but string of length 0 found But with cPickle, the result is: Traceback (most recent call last): File "D:\src\misc\CDROM\python\pickletest.py", line 68, in ? tree=load( f ) EOFError Small changes in the datastructure lead to different results: try changing the line if level<4: to if level<1: Now cPickle works, but not pickle. This happens with Python 2.3.3 and 2.3.5 on Windows XP. ---------------------------------------------------------------------- >Comment By: Heiko Selber (drhok) Date: 2005-04-22 20:54 Message: Logged In: YES user_id=865975 Ouch! You're right, I failed to specify binary mode. Sorry, closing. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-22 17:01 Message: Logged In: YES user_id=31435 How did you open the file? You must open files holding pickle data in binary mode on Windows (and should open them in binary mode on all platforms). IOW, you must do f = open(whatever, 'wb') when opening a file to which you write a pickle, and must use f = open(whatever, 'rb') when opening a file from which you want to read a pickle. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188175&group_id=5470 From noreply at sourceforge.net Sat Apr 23 05:42:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 23 05:42:59 2005 Subject: [ python-Bugs-1173773 ] multiple broken links in profiler docs Message-ID: Bugs item #1173773, was opened at 2005-03-30 18:19 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173773&group_id=5470 Category: Documentation Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Ilya Sandler (isandler) Assigned to: Nobody/Anonymous (nobody) Summary: multiple broken links in profiler docs Initial Comment: I'm looking at http://www.python.org/doc/current/lib/lib.html and multiple links in profiler section seem to be broken: 10.1 Introduction... (Error 404) http://www.python.org/doc/current/lib/node434.html 10.2 How is this profiler different: (Error 404) http://www.python.org/doc/current/lib/node435.html 10.3. What's is deterministic... (leads to wrong page) http://www.python.org/doc/current/lib/node437.html 10.8 Extensions... (Error 404) http://www.python.org/doc/current/lib/node442.html 10.10.1 Command Line Interface (leads to wrong page) http://www.python.org/doc/current/lib/node448.html 10.10.2 Examples (Error 404) http://www.python.org/doc/current/lib/node449.html ---------------------------------------------------------------------- >Comment By: Ilya Sandler (isandler) Date: 2005-04-22 20:42 Message: Logged In: YES user_id=971153 As of 04/22/05 all links appear to work.. So the bug should be closed ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1173773&group_id=5470 From noreply at sourceforge.net Sat Apr 23 06:02:57 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 23 06:03:00 2005 Subject: [ python-Bugs-415492 ] Compiler generates relative filenames Message-ID: Bugs item #415492, was opened at 2001-04-11 11:52 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=415492&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Paul Prescod (prescod) Assigned to: Nobody/Anonymous (nobody) Summary: Compiler generates relative filenames Initial Comment: Make a file called "test.py" ---- import pack print pack.func.func_code.co_filename ----- Make a directory called "pack". Put a file in it called "__init__.py" with the contents: def func(): pass Now run test.py. It will compile a relative path into these modules. Now you can change to any directory on the system and run test.py and it will return the original relative path. The problem is that the relative path is compiled into the .pyc. It should be an absolute path. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-22 21:02 Message: Logged In: YES user_id=971153 I tried to reproduce the problem with python 2.4 and I'm getting the absolute path.... So, I guess, the bug can be closed ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2002-04-25 14:08 Message: Logged In: YES user_id=86307 I just ran into this today when trying to find out why pydoc wasn't working for a module of mine: inspect.getmodule calls os.abspath with the filename found in the code object of global functions. If the cwd is different than when the code object was compiled, inspect.getmodule fails. Anyway, it looks to me like most of these relative paths could be caught in the find_module function (of import.c) if, when given an empty string for a path (while iterating over sys.path), find_module called getcwd() and used that instead of the empty string. Of course, this assumes that (on all platforms) opening a bare filename means open the file with that name in the current directory (is that a valid assumption?). Also, it appears from posixmodule.c that getcwd may not always be available. Does this sound like a reasonable idea? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:10 Message: Logged In: YES user_id=6380 Why on earth was this assigned to Paul? He's not going to make progress. Assigning back to nobody. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-04-13 14:15 Message: Logged In: YES user_id=31392 Just to clarify. The compiler in question is the builtin compiler. It generates absolute path names unless the .py file is in the current working directory. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-04-13 07:12 Message: Logged In: YES user_id=31392 I'm not clear on what the rules for co_filename are, but it looks like the absolute path is only used for package imports. If you cd into the package directory, run python -c "import __init__", and then later import the package the path name is relative. The compiler package isn't connected to the import mechanism, so there's no way for it to know whether it is compiling a module or a package. I don't think there's a way to do anything better. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=415492&group_id=5470 From noreply at sourceforge.net Sat Apr 23 18:45:13 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 23 18:45:33 2005 Subject: [ python-Bugs-1188637 ] Python 2.4 Not Recognized by Any Programs Message-ID: Bugs item #1188637, was opened at 2005-04-23 11:45 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188637&group_id=5470 Category: Installation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Yoshi Nagasaki (gr1ever) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.4 Not Recognized by Any Programs Initial Comment: Hi, I've tried installing Python 2.4 on two different Windows machines, one running 98 and the other running XP Pro. In both cases, after installing it, none of the programs that use python recognize that it has been installed. These programs include WinCVS and Blender and perhaps others. Only installations of older versions of Python are recognized by the programs (I think I'm using 2.3). Any suggestions on what might be the problem? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188637&group_id=5470 From noreply at sourceforge.net Sat Apr 23 18:46:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 23 18:46:48 2005 Subject: [ python-Bugs-1175396 ] codecs.readline sometimes removes newline chars Message-ID: Bugs item #1175396, was opened at 2005-04-02 06:14 Message generated for change (Comment added) made by glchapman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: Accepted Priority: 5 Submitted By: Irmen de Jong (irmen) Assigned to: Walter D?rwald (doerwalter) Summary: codecs.readline sometimes removes newline chars Initial Comment: In Python 2.4.1 i observed a new bug in codecs.readline, it seems that with certain inputs it removes newline characters from the end of the line.... Probably related to bug #1076985 (Incorrect behaviour of StreamReader.readline leads to crash) and bug #1098990 codec readline() splits lines apart (both with status closed) so I'm assigning this to Walter. See the attached files that demonstrate the problem. Reproduced with Python 2.4.1 on windows XP and on Linux. The problem does not occur with Python 2.4. (btw, it seems bug #1076985 was fixed in python 2.4.1, but the other one (#1098990) not? ) ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-23 08:46 Message: Logged In: YES user_id=86307 > 1) How do we handle the problem of a truncated line, if the > data comes from the charbuffer instead of being read from > the stream? My suggestion is to make the top of the loop look like: while True: havecharbuffer = bool(self.charbuffer) And then the break condition (when no line break found) should be: # we didn't get anything or this was our only try if not data or (size is not None and not havecharbuffer): (too many negatives in that). Anyway, the idea is that, if size specified, there will be at most one read of the underlying stream (using the size). So if you enter the loop with a charbuffer, and that charbuffer does not have a line break, then you redo the loop once (the second time it will break, because havecharbuffer will be False). Also, not sure about this, but should the size parameter default to -1 (to keep it in sync with read)? As to issue 2, it looks like it should be possible to get the line number right, because the UnicodeDecodeError exception object has all the necessary information in it (including the original string). I think this should be done by fp_readl (in tokenizer.c). By the way, using a findlinebreak function (using sre) turns out to be slower than splitting/joining when no size is specified (which I assume will be the overwhelmingly most common case), so that turned out to be a bad idea on my part. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-21 13:55 Message: Logged In: YES user_id=89016 OK, I've checked in the following: Lib/codecs.py 1.44 Lib/test/test_codecs.py 1.23 Lib/codecs.py 1.35.2.7 Lib/test/test_codecs.py 1.15.2.5 with the following changes as suggested by glchapman: If a chunk read has a trailing "\r", read one more character even if the user has passed a size parameter. Remove the atcr logic. This should fix most of the problems. There are three open issues: 1) How do we handle the problem of a truncated line, if the data comes from the charbuffer instead of being read from the stream? 2) How do we handle error reporting? The buffering code might read more data than the current line. If this data has a decoding error the caller might report a wrong line number. (See bug #1178484) 3) Fixing the tokenizer. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-15 10:16 Message: Logged In: YES user_id=89016 The current readline() is implemented so that even if the complete line ending (i.e. '\r\n') can't be read within size bytes, at least something that might work as a line end (i.e. '\r') is available at the end of the string, so that the user knowns that the line is complete. The atcr members makes sure that the '\n' that is read next isn't misinterpreted as another line. Unfortunately the import mechanisn doesn't work that way: It demands a '\n' as line terminator and will continue reading if it only finds the '\r'. This means that the '\n' will be skipped and the import mechanisn treats those two lines as one. IMHO the best solution would be the read the extra character even if size is None, as glchapman suggested, so the above situation would really only happen if the last character from the stream is '\r'. I think the tokenizer should be able to survive this. What it didn't survive in 2.4 was that readline() tried to get it's hand on a complete line even if the length of the line was way bigger than the size passed in. If we remove the "size is None" restriction from the current code, then I think we should remove the atcr logic too, because it could only happen under exceptional circumstances and the old handling might be better for those users that don't recognize '\r' as a line end. But in any case the tokenizer should be fixed to be able to handle any line length returned from readline(). I'd really like to get a review by Martin v. L?wis of glchapman's patch #1101726. Of course the simplest solution would be: "If you want a complete line, don't pass a size parameter to readline()". I don't know if it would make sense to change the PEP263 machinery to support this. The other problem is if readline() returns data from the charbuffer. I don't really see how this can be fixed. We might call read() with a size parameter even if there are characters in the charbuffer, but what happens if the user calls readline(size=100000) first and then readline(size=10)? The first readline() call might put a very long string into the charbuffer and then the second call will return a unicode string whose length is in no way correlated to the size parameter. (This happens even even with the current code of course, so the message should be: Don't trust the size parameter, it only tells the underlying stream how many bytes to read (approximately), it's doesn't tell you *anything* about the amount of data returned.). This was different in 2.3.x, because the readline() method of the underlying stream was used, which handled this properly, but only worked if an encoded linefeed was recognizable as a normal 8bit '\n' linefeed by the underlying readline() (i.e. the UTF-16 encodings didn't work). ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-14 15:25 Message: Logged In: YES user_id=86307 I think the foo2.py from 1163244 is probably the same bug; at any rate, the reason for it is that a \r is at the beginning of the last line when read in by decoding_fgets. I have simpler test file which shows the bug which I'll email to Walter (you basically just have to get a \r as the last character in the block read by StreamReader, so that atcr will be true). The problem is caused by StreamReader.readline doing: if self.atcr and data.startswith(u"\n"): data = data[1:] since the tokenizer relies on '\n' as the line break character, but it will never see the '\n' removed by the above. FWIW (not much), I think the 2.4 StreamReader.readline actually made more sense than the current code, although a few changes would seem useful (see below). I don't think it is particularly useful to treat the size parameter as a fixed maximum number of bytes to read, since the number of bytes read has no fixed relationship to the number of decoded unicode characters (and also, in the case of the tokenizer, no fixed relationship to the number of bytes of encoded utf8). Also, given the current code, the size parameter is effectively ignored if there is a charbuffer: if you have 5 characters sitting in the charbuffer and use a size of 0x1FF, you only get back the 5 characters, even if they do not end in a linebreak. For the tokenizer, this means an unnecessary PyMem_RESIZE and an extra call to decoding_readline roughly every BUFSIZ bytes in the file (since the tokenizer assumes failure to fetch a complete line means its buffer is too small, whereas in fact it was caused by an incomplete line being stored in the StreamReader's charbuffer). As to changes from 2.4, if the unicode object were to add a findlinebreak method which returns the index of the first character for which Py_UNICODE_ISLINEBREAK is true, readline could use that instead of find("\n"). If it used such a method, readline would also need to explicitly handle a "\r\n" sequence, including a potential read(1) if a '\r' appears at the end of the data (in the case where size is not None). Of course, one problem with that idea is it requires a new method, which may not be allowed until 2.5, and the 2.4.1 behavior definitely needs to be fixed some way. (Interestingly, it looks to me like sre has everything necessary for searching for unicode linebreaks except syntax with which to express the idea in a pattern (maybe I'm missing something, but I can't find a way to get a compiled pattern to emit CATEGORY_UNI_LINEBREAK).) ---------------------------------------------------------------------- Comment By: Michal Rydlo (mmm) Date: 2005-04-14 14:04 Message: Logged In: YES user_id=65460 foo2.py from #1163244 fails to import. Not being expert in Python internals I hope it is due to this bug. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-11 13:42 Message: Logged In: YES user_id=89016 OK, I'm reopening to bug report. I didn't manage to install pythondoc. cElementTree complains about: No such file or directory: './pyconfig.h'. Can you provide a simple Python file that fails when imported? ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2005-04-09 14:47 Message: Logged In: YES user_id=86307 Sorry to comment on a closed report, but perhaps this fix should not be limited only to cases where size is None. Today, I ran into a spurious syntax error when trying to import pythondoc (from http://effbot.org/downloads/pythondoc-2.1b3-20050325.zip). It turned out a \r was ending up in what looked to the parser like the middle of a line, presumably because a \n was dropped. Anyway, I applied the referenced patch to 2.4.1, except I left out the "size is None" condition, since I knew the tokenizer passes in a size param. With that change pythondoc import successfully. (Also, I just ran the test suite and nothing broke.) Since the size parameter is already documented as being passed to StreamReader.read (in codecs.py -- the HTML documentation needs to be updated), and since StreamReader.read says size is an approximate maximum, perhaps it's OK to read one extra byte. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2005-04-04 14:01 Message: Logged In: YES user_id=89016 Checked in a fix as: Lib/codecs.py 1.42/1.43 Lib/test/test_codecs.py 1.22 Lib/codecs.py 1.35.2.6 Lib/test/test_codecs.py 1.15.2.4 Are you really sure, that the fix for #1098990 is not in 2.4.1? According to the tracker for #1098990 the fix was in lib/codecs.py revision 1.35.2.2 and revision 1.35.2.3 is the one that got the r241c1 tag, so the fix should be in 2.4.1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1175396&group_id=5470 From noreply at sourceforge.net Mon Apr 25 03:18:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 03:18:59 2005 Subject: [ python-Bugs-1166714 ] [ast branch] fatal error when compiling test_bool.py Message-ID: Bugs item #1166714, was opened at 2005-03-20 00:14 Message generated for change (Comment added) made by jpe You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: John Ehresman (jpe) Assigned to: Brett Cannon (bcannon) Summary: [ast branch] fatal error when compiling test_bool.py Initial Comment: When using the ast-branch compiler, python exits with a fatal error when compiling test_bool.py or the symplified test case that I'll attach. The immediate problem is that when the compiler is in make_closure after compiling the lambda, self is found to be a LOCAL reference type and make_closure assumes it's a FREE reference if it's not a CELL reference in the enclosing scope. I don't know if self is supposed to be a LOCAL reference or if make_closure should handle LOCAL references. ---------------------------------------------------------------------- >Comment By: John Ehresman (jpe) Date: 2005-04-25 01:18 Message: Logged In: YES user_id=22785 Yes, it should be PyDict_Update(free, newfree) like it is in the current CVS. The unicode failures should be fixed by compiling and using tokenizer.c rather than tokenizer_pgen.c, which is mentioned in the comment for patch 1170272 but didn't get changed when the patch was applied. I'm pretty sure tokenizer_pgen.c is not needed. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-19 05:26 Message: Logged In: YES user_id=357491 John, on line 45 of the patch, which becomes line 580 in Python/symtable.c, you call PyDict_Update() with three arguments. Is 'local' supposed to be an argument, or is 'free' the erroneous one? If I change it so that 'local' is removed then the tests die on test_unicode (although a ton of other tests fail). ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-23 01:22 Message: Logged In: YES user_id=22785 Forgot to add the patch ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-23 01:21 Message: Logged In: YES user_id=22785 The attached patch seems to fix this by creating a dictionary for each scope's free vars and only propagating up the ones that aren't satisfied by a binding. With this and the other patches, the test suite runs w/o segfaulting up through test_unicode. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 21:37 Message: Logged In: YES user_id=22785 The issue seems to be that in symtable_analyze, a free dictionary is created for the entire module and if a name is found to be free in any function, it is not converted from a LOCAL to a CELL is any context seen afterwards. I think the free variables need to be recorded on a function-by-function basis. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-03-22 20:34 Message: Logged In: YES user_id=357491 An short example of the failure:: def outer(a): def first(): a def last(a): pass This leads to a fatal Python error. The issue is having a being a local or parameter (can make 'a' be 'a = 1' and not a parameter and still have it fail), the first closure reference it, and then the second one use it as a parameter. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 From noreply at sourceforge.net Mon Apr 25 04:08:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 04:08:05 2005 Subject: [ python-Bugs-1189216 ] zipfile module and 2G boundary Message-ID: Bugs item #1189216, was opened at 2005-04-24 22:08 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189216&group_id=5470 Category: Python Library Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Bob Ippolito (etrepum) Assigned to: Nobody/Anonymous (nobody) Summary: zipfile module and 2G boundary Initial Comment: The zipfile module currently can not handle archives that have file headers that begin past the 2**31 byte boundary. This is really bug #679953 all over again -- that fix didn't solve all of the problem. Patch to CVS HEAD attached, backport candidate to 2.4.2 and 2.3.6. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189216&group_id=5470 From noreply at sourceforge.net Mon Apr 25 04:27:42 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 04:27:44 2005 Subject: [ python-Bugs-1186345 ] [AST] assert failure on ``eval("u'\Ufffffffe'")`` Message-ID: Bugs item #1186345, was opened at 2005-04-19 16:24 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] assert failure on ``eval("u'\Ufffffffe'")`` Initial Comment: Isolated the failure of test_unicode to be because of the test of ``eval("u'\Ufffffffe'")``. What is odd is that the Unicode string works fine as a literal at the intepreter prompt. Somehow eval() is triggering this problem. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-24 19:27 Message: Logged In: YES user_id=357491 jpe suggests looking at the comment of tracker #1170272; use tokenizer.c instead of tokenizer_pgen.c (this is mentioned in bug #1166714). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 From noreply at sourceforge.net Mon Apr 25 04:28:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 04:28:23 2005 Subject: [ python-Bugs-1166714 ] [ast branch] fatal error when compiling test_bool.py Message-ID: Bugs item #1166714, was opened at 2005-03-19 16:14 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: John Ehresman (jpe) Assigned to: Brett Cannon (bcannon) Summary: [ast branch] fatal error when compiling test_bool.py Initial Comment: When using the ast-branch compiler, python exits with a fatal error when compiling test_bool.py or the symplified test case that I'll attach. The immediate problem is that when the compiler is in make_closure after compiling the lambda, self is found to be a LOCAL reference type and make_closure assumes it's a FREE reference if it's not a CELL reference in the enclosing scope. I don't know if self is supposed to be a LOCAL reference or if make_closure should handle LOCAL references. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-24 19:28 Message: Logged In: YES user_id=357491 OK, added a comment in bug #1186345 to this fact. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-04-24 18:18 Message: Logged In: YES user_id=22785 Yes, it should be PyDict_Update(free, newfree) like it is in the current CVS. The unicode failures should be fixed by compiling and using tokenizer.c rather than tokenizer_pgen.c, which is mentioned in the comment for patch 1170272 but didn't get changed when the patch was applied. I'm pretty sure tokenizer_pgen.c is not needed. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-18 22:26 Message: Logged In: YES user_id=357491 John, on line 45 of the patch, which becomes line 580 in Python/symtable.c, you call PyDict_Update() with three arguments. Is 'local' supposed to be an argument, or is 'free' the erroneous one? If I change it so that 'local' is removed then the tests die on test_unicode (although a ton of other tests fail). ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 17:22 Message: Logged In: YES user_id=22785 Forgot to add the patch ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 17:21 Message: Logged In: YES user_id=22785 The attached patch seems to fix this by creating a dictionary for each scope's free vars and only propagating up the ones that aren't satisfied by a binding. With this and the other patches, the test suite runs w/o segfaulting up through test_unicode. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 13:37 Message: Logged In: YES user_id=22785 The issue seems to be that in symtable_analyze, a free dictionary is created for the entire module and if a name is found to be free in any function, it is not converted from a LOCAL to a CELL is any context seen afterwards. I think the free variables need to be recorded on a function-by-function basis. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-03-22 12:34 Message: Logged In: YES user_id=357491 An short example of the failure:: def outer(a): def first(): a def last(a): pass This leads to a fatal Python error. The issue is having a being a local or parameter (can make 'a' be 'a = 1' and not a parameter and still have it fail), the first closure reference it, and then the second one use it as a parameter. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 From noreply at sourceforge.net Mon Apr 25 04:35:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 04:35:55 2005 Subject: [ python-Bugs-1186345 ] [AST] assert failure on ``eval("u'\Ufffffffe'")`` Message-ID: Bugs item #1186345, was opened at 2005-04-19 23:24 Message generated for change (Comment added) made by jpe You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] assert failure on ``eval("u'\Ufffffffe'")`` Initial Comment: Isolated the failure of test_unicode to be because of the test of ``eval("u'\Ufffffffe'")``. What is odd is that the Unicode string works fine as a literal at the intepreter prompt. Somehow eval() is triggering this problem. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-04-25 02:35 Message: Logged In: YES user_id=22785 Also look at patch 1189210, which probably should have just been attached to this bug. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-25 02:27 Message: Logged In: YES user_id=357491 jpe suggests looking at the comment of tracker #1170272; use tokenizer.c instead of tokenizer_pgen.c (this is mentioned in bug #1166714). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 From noreply at sourceforge.net Mon Apr 25 07:00:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 07:00:44 2005 Subject: [ python-Bugs-1189248 ] Seg Fault when compiling small program Message-ID: Bugs item #1189248, was opened at 2005-04-24 22:00 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189248&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Reginald B. Charney (rcharney) Assigned to: Nobody/Anonymous (nobody) Summary: Seg Fault when compiling small program Initial Comment: I am using SuSE 9.2. When trying to compile/interpret the attached file, the python interpreter seg faults with no other information. Please note: the attached test file does not end in a new line. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189248&group_id=5470 From noreply at sourceforge.net Mon Apr 25 08:42:41 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 08:42:43 2005 Subject: [ python-Bugs-1189248 ] Seg Fault when compiling small program Message-ID: Bugs item #1189248, was opened at 2005-04-24 22:00 Message generated for change (Comment added) made by rcharney You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189248&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Reginald B. Charney (rcharney) Assigned to: Nobody/Anonymous (nobody) Summary: Seg Fault when compiling small program Initial Comment: I am using SuSE 9.2. When trying to compile/interpret the attached file, the python interpreter seg faults with no other information. Please note: the attached test file does not end in a new line. ---------------------------------------------------------------------- >Comment By: Reginald B. Charney (rcharney) Date: 2005-04-24 23:42 Message: Logged In: YES user_id=411372 It turns out that the seg fault was caused by the first line of the program: # coding: evil It seems that specifying an invalid character encoding causes a seg fault. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189248&group_id=5470 From noreply at sourceforge.net Mon Apr 25 09:56:17 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 09:56:20 2005 Subject: [ python-Bugs-1189248 ] Seg Fault when compiling small program Message-ID: Bugs item #1189248, was opened at 2005-04-25 06:00 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189248&group_id=5470 Category: Python Interpreter Core Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Reginald B. Charney (rcharney) Assigned to: Nobody/Anonymous (nobody) Summary: Seg Fault when compiling small program Initial Comment: I am using SuSE 9.2. When trying to compile/interpret the attached file, the python interpreter seg faults with no other information. Please note: the attached test file does not end in a new line. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-25 08:56 Message: Logged In: YES user_id=6656 I think this is fixed in Python 2.4.1... what version are you using? ---------------------------------------------------------------------- Comment By: Reginald B. Charney (rcharney) Date: 2005-04-25 07:42 Message: Logged In: YES user_id=411372 It turns out that the seg fault was caused by the first line of the program: # coding: evil It seems that specifying an invalid character encoding causes a seg fault. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189248&group_id=5470 From noreply at sourceforge.net Mon Apr 25 11:01:08 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 11:01:10 2005 Subject: [ python-Bugs-1189330 ] LINKCC incorrect Message-ID: Bugs item #1189330, was opened at 2005-04-25 11:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189330&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Christoph Ludwig (cludwig) Assigned to: Nobody/Anonymous (nobody) Summary: LINKCC incorrect Initial Comment: I configured Python 2.4.1 as follows ../Python-2.4.1/configure \ --prefix=/home/cludwig/C++/gcc4.0/Python-2.4.1 \ --with-cxx=/opt/gcc/gcc-4.0.0/bin/g++ --enable-shared \ --enable-unicode --with-signal-module \ --with-universal-newlines --with-doc-strings on a i686-pc-linux-gnu system. make fails when linking the python binariy due to an undefined reference to `__gxx_personality_v0'. In fact, configure set CC= gcc -pthread CXX= /opt/gcc/gcc-4.0.0/bin/g++ -pthread LINKCC= $(PURIFY) $(CC) but the python executable needs to be linked with $(CXX). (Note the `--with-cxx' option in the configure command line.) I did not observe this problem with Python 2.4.0 / gcc 3.4.2. This seems to be a regression w.r.t. PR #569668 that was closed as fixed on 2002-12-03. I can submit config.log and the output of make on request. Regards Christoph ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189330&group_id=5470 From noreply at sourceforge.net Mon Apr 25 11:19:57 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 11:20:00 2005 Subject: [ python-Bugs-1189337 ] LINKCC incorrect Message-ID: Bugs item #1189337, was opened at 2005-04-25 11:19 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189337&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Christoph Ludwig (cludwig) Assigned to: Nobody/Anonymous (nobody) Summary: LINKCC incorrect Initial Comment: I configured Python 2.4.1 as follows ../Python-2.4.1/configure \ --prefix=/home/cludwig/C++/gcc4.0/Python-2.4.1 \ --with-cxx=/opt/gcc/gcc-4.0.0/bin/g++ --enable-shared \ --enable-unicode --with-signal-module \ --with-universal-newlines --with-doc-strings on a i686-pc-linux-gnu system. make fails when linking the python binariy due to an undefined reference to `__gxx_personality_v0'. In fact, configure set CC= gcc -pthread CXX= /opt/gcc/gcc-4.0.0/bin/g++ -pthread LINKCC= $(PURIFY) $(CC) but the python executable needs to be linked with $(CXX). (Note the `--with-cxx' option in the configure command line.) I did not observe this problem with Python 2.4.0 / gcc 3.4.2. This seems to be a regression w.r.t. PR #569668 that was closed as fixed on 2002-12-03. I can submit config.log and the output of make on request. Regards Christoph ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189337&group_id=5470 From noreply at sourceforge.net Mon Apr 25 11:25:11 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 11:25:13 2005 Subject: [ python-Bugs-1189337 ] LINKCC incorrect Message-ID: Bugs item #1189337, was opened at 2005-04-25 11:19 Message generated for change (Comment added) made by cludwig You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189337&group_id=5470 Category: Build Group: Python 2.4 >Status: Closed Resolution: None Priority: 5 Submitted By: Christoph Ludwig (cludwig) Assigned to: Nobody/Anonymous (nobody) Summary: LINKCC incorrect Initial Comment: I configured Python 2.4.1 as follows ../Python-2.4.1/configure \ --prefix=/home/cludwig/C++/gcc4.0/Python-2.4.1 \ --with-cxx=/opt/gcc/gcc-4.0.0/bin/g++ --enable-shared \ --enable-unicode --with-signal-module \ --with-universal-newlines --with-doc-strings on a i686-pc-linux-gnu system. make fails when linking the python binariy due to an undefined reference to `__gxx_personality_v0'. In fact, configure set CC= gcc -pthread CXX= /opt/gcc/gcc-4.0.0/bin/g++ -pthread LINKCC= $(PURIFY) $(CC) but the python executable needs to be linked with $(CXX). (Note the `--with-cxx' option in the configure command line.) I did not observe this problem with Python 2.4.0 / gcc 3.4.2. This seems to be a regression w.r.t. PR #569668 that was closed as fixed on 2002-12-03. I can submit config.log and the output of make on request. Regards Christoph ---------------------------------------------------------------------- >Comment By: Christoph Ludwig (cludwig) Date: 2005-04-25 11:25 Message: Logged In: YES user_id=1266029 Oops, this is a duplicate of PR #1189330. It was inadvertently submitted when I clicked on "reload" in my browser whereupon it resent the formula data. Sorry about that. Christoph ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189337&group_id=5470 From noreply at sourceforge.net Mon Apr 25 11:27:08 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 11:27:11 2005 Subject: [ python-Bugs-1189337 ] LINKCC incorrect Message-ID: Bugs item #1189337, was opened at 2005-04-25 11:19 Message generated for change (Settings changed) made by cludwig You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189337&group_id=5470 Category: Build Group: Python 2.4 Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Christoph Ludwig (cludwig) Assigned to: Nobody/Anonymous (nobody) Summary: LINKCC incorrect Initial Comment: I configured Python 2.4.1 as follows ../Python-2.4.1/configure \ --prefix=/home/cludwig/C++/gcc4.0/Python-2.4.1 \ --with-cxx=/opt/gcc/gcc-4.0.0/bin/g++ --enable-shared \ --enable-unicode --with-signal-module \ --with-universal-newlines --with-doc-strings on a i686-pc-linux-gnu system. make fails when linking the python binariy due to an undefined reference to `__gxx_personality_v0'. In fact, configure set CC= gcc -pthread CXX= /opt/gcc/gcc-4.0.0/bin/g++ -pthread LINKCC= $(PURIFY) $(CC) but the python executable needs to be linked with $(CXX). (Note the `--with-cxx' option in the configure command line.) I did not observe this problem with Python 2.4.0 / gcc 3.4.2. This seems to be a regression w.r.t. PR #569668 that was closed as fixed on 2002-12-03. I can submit config.log and the output of make on request. Regards Christoph ---------------------------------------------------------------------- Comment By: Christoph Ludwig (cludwig) Date: 2005-04-25 11:25 Message: Logged In: YES user_id=1266029 Oops, this is a duplicate of PR #1189330. It was inadvertently submitted when I clicked on "reload" in my browser whereupon it resent the formula data. Sorry about that. Christoph ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189337&group_id=5470 From noreply at sourceforge.net Mon Apr 25 16:51:17 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 16:51:20 2005 Subject: [ python-Bugs-1189525 ] file.write(x) where len(x) > 64*1024**2 is unreliable Message-ID: Bugs item #1189525, was opened at 2005-04-25 16:51 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Martin Gfeller (gfe) Assigned to: Nobody/Anonymous (nobody) Summary: file.write(x) where len(x) > 64*1024**2 is unreliable Initial Comment: Large file writes (over 64MB) are unreliable under Windows. On my Windows 2000 SP4, it succeeds when writing to a file on a local driver, or on a network drive. Howver, if I map the share of the local drive c$ as another drive, the same write fails with: IOError: [Errno 22] Invalid argument Filemon (www.sysinternals.com) shows that this is a STATUS_INSUFFICIENT_RESOURCES returned by IRP_MJ_WRITE. Although this is probably a Windows bug, my suggestion is that Python should abstract from such problems, and break the writes into suitable chunks. Best regards, Martin ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 From noreply at sourceforge.net Mon Apr 25 18:48:06 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 18:48:12 2005 Subject: [ python-Bugs-1189525 ] file.write(x) where len(x) > 64*1024**2 is unreliable Message-ID: Bugs item #1189525, was opened at 2005-04-25 10:51 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 Category: Windows >Group: 3rd Party >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Martin Gfeller (gfe) Assigned to: Nobody/Anonymous (nobody) Summary: file.write(x) where len(x) > 64*1024**2 is unreliable Initial Comment: Large file writes (over 64MB) are unreliable under Windows. On my Windows 2000 SP4, it succeeds when writing to a file on a local driver, or on a network drive. Howver, if I map the share of the local drive c$ as another drive, the same write fails with: IOError: [Errno 22] Invalid argument Filemon (www.sysinternals.com) shows that this is a STATUS_INSUFFICIENT_RESOURCES returned by IRP_MJ_WRITE. Although this is probably a Windows bug, my suggestion is that Python should abstract from such problems, and break the writes into suitable chunks. Best regards, Martin ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-25 12:48 Message: Logged In: YES user_id=31435 Python's file.write() just calls the platform C's fwrite(), and MS's fwrite() docs don't admit to any size limitations. If this is a documented bug on Windows, then your complaint is with Microsoft. If it's not documented, how is anyone supposed to guess what "suitable" means in all cases? IOW, this report is either misdirected, or impossible to fix sanely. Sorry, but I'm closing as "3rd Party, WontFix" on these grounds. If you care enough to research it in depth and create a patch, that may be a different story. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 From noreply at sourceforge.net Mon Apr 25 19:03:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 19:04:04 2005 Subject: [ python-Bugs-1189525 ] file.write(x) where len(x) > 64*1024**2 is unreliable Message-ID: Bugs item #1189525, was opened at 2005-04-25 16:51 Message generated for change (Comment added) made by gfe You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 Category: Windows Group: 3rd Party Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Martin Gfeller (gfe) Assigned to: Nobody/Anonymous (nobody) Summary: file.write(x) where len(x) > 64*1024**2 is unreliable Initial Comment: Large file writes (over 64MB) are unreliable under Windows. On my Windows 2000 SP4, it succeeds when writing to a file on a local driver, or on a network drive. Howver, if I map the share of the local drive c$ as another drive, the same write fails with: IOError: [Errno 22] Invalid argument Filemon (www.sysinternals.com) shows that this is a STATUS_INSUFFICIENT_RESOURCES returned by IRP_MJ_WRITE. Although this is probably a Windows bug, my suggestion is that Python should abstract from such problems, and break the writes into suitable chunks. Best regards, Martin ---------------------------------------------------------------------- >Comment By: Martin Gfeller (gfe) Date: 2005-04-25 19:03 Message: Logged In: YES user_id=884167 As you confirm that fwrite() has no documented size limitations, it must be a Windows bug. So I will need to find a more practical solution instead of assuming that Python will protect me from the rough edges of OS. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-25 18:48 Message: Logged In: YES user_id=31435 Python's file.write() just calls the platform C's fwrite(), and MS's fwrite() docs don't admit to any size limitations. If this is a documented bug on Windows, then your complaint is with Microsoft. If it's not documented, how is anyone supposed to guess what "suitable" means in all cases? IOW, this report is either misdirected, or impossible to fix sanely. Sorry, but I'm closing as "3rd Party, WontFix" on these grounds. If you care enough to research it in depth and create a patch, that may be a different story. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 From noreply at sourceforge.net Mon Apr 25 19:14:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 19:15:01 2005 Subject: [ python-Bugs-1189525 ] file.write(x) where len(x) > 64*1024**2 is unreliable Message-ID: Bugs item #1189525, was opened at 2005-04-25 14:51 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 Category: Windows Group: 3rd Party Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Martin Gfeller (gfe) Assigned to: Nobody/Anonymous (nobody) Summary: file.write(x) where len(x) > 64*1024**2 is unreliable Initial Comment: Large file writes (over 64MB) are unreliable under Windows. On my Windows 2000 SP4, it succeeds when writing to a file on a local driver, or on a network drive. Howver, if I map the share of the local drive c$ as another drive, the same write fails with: IOError: [Errno 22] Invalid argument Filemon (www.sysinternals.com) shows that this is a STATUS_INSUFFICIENT_RESOURCES returned by IRP_MJ_WRITE. Although this is probably a Windows bug, my suggestion is that Python should abstract from such problems, and break the writes into suitable chunks. Best regards, Martin ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2005-04-25 17:14 Message: Logged In: YES user_id=849994 Well, when there's no bug documented on the Windows side, Python cannot work around it. ---------------------------------------------------------------------- Comment By: Martin Gfeller (gfe) Date: 2005-04-25 17:03 Message: Logged In: YES user_id=884167 As you confirm that fwrite() has no documented size limitations, it must be a Windows bug. So I will need to find a more practical solution instead of assuming that Python will protect me from the rough edges of OS. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-25 16:48 Message: Logged In: YES user_id=31435 Python's file.write() just calls the platform C's fwrite(), and MS's fwrite() docs don't admit to any size limitations. If this is a documented bug on Windows, then your complaint is with Microsoft. If it's not documented, how is anyone supposed to guess what "suitable" means in all cases? IOW, this report is either misdirected, or impossible to fix sanely. Sorry, but I'm closing as "3rd Party, WontFix" on these grounds. If you care enough to research it in depth and create a patch, that may be a different story. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 From noreply at sourceforge.net Mon Apr 25 19:18:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 19:18:19 2005 Subject: [ python-Bugs-1189525 ] file.write(x) where len(x) > 64*1024**2 is unreliable Message-ID: Bugs item #1189525, was opened at 2005-04-25 10:51 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 Category: Windows Group: 3rd Party Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Martin Gfeller (gfe) Assigned to: Nobody/Anonymous (nobody) Summary: file.write(x) where len(x) > 64*1024**2 is unreliable Initial Comment: Large file writes (over 64MB) are unreliable under Windows. On my Windows 2000 SP4, it succeeds when writing to a file on a local driver, or on a network drive. Howver, if I map the share of the local drive c$ as another drive, the same write fails with: IOError: [Errno 22] Invalid argument Filemon (www.sysinternals.com) shows that this is a STATUS_INSUFFICIENT_RESOURCES returned by IRP_MJ_WRITE. Although this is probably a Windows bug, my suggestion is that Python should abstract from such problems, and break the writes into suitable chunks. Best regards, Martin ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-25 13:18 Message: Logged In: YES user_id=31435 Python hides all the rough edges it can, where "can" means a combination of knowing in advance precisely where rough edges are, and of workarounds being practical. In this case I'd _guess_ that it really has nothing to do with fwrite(), it has to do with limitations introduced by specific ways of mapping drives. For all I know (and I don't know, one way or the other), Microsoft does document such limitations somewhere. If so, they'll say it's a limitation, and not a bug. That's common enough. It may even be that there's some non-obvious knob you could change on Windows to increase whatever resource it is that you're running out of when writing > 64MB to a map of a share of the local c$ drive. But Python isn't the OS regardless, and there are hundreds of OS and platform C limitations Python can't sanely hide. This looks like one of 'em. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2005-04-25 13:14 Message: Logged In: YES user_id=849994 Well, when there's no bug documented on the Windows side, Python cannot work around it. ---------------------------------------------------------------------- Comment By: Martin Gfeller (gfe) Date: 2005-04-25 13:03 Message: Logged In: YES user_id=884167 As you confirm that fwrite() has no documented size limitations, it must be a Windows bug. So I will need to find a more practical solution instead of assuming that Python will protect me from the rough edges of OS. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-25 12:48 Message: Logged In: YES user_id=31435 Python's file.write() just calls the platform C's fwrite(), and MS's fwrite() docs don't admit to any size limitations. If this is a documented bug on Windows, then your complaint is with Microsoft. If it's not documented, how is anyone supposed to guess what "suitable" means in all cases? IOW, this report is either misdirected, or impossible to fix sanely. Sorry, but I'm closing as "3rd Party, WontFix" on these grounds. If you care enough to research it in depth and create a patch, that may be a different story. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189525&group_id=5470 From noreply at sourceforge.net Mon Apr 25 23:00:42 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 23:00:46 2005 Subject: [ python-Bugs-1189811 ] pydoc may hide non-private doc strings. Message-ID: Bugs item #1189811, was opened at 2005-04-25 16:00 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189811&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: J Livingston (jlivingston) Assigned to: Nobody/Anonymous (nobody) Summary: pydoc may hide non-private doc strings. Initial Comment: I am using Python version 2.3.4, pydoc version 1.86.8.1 on WinXP SP2. pydoc's visiblename() method indicates "Private names are hidden, but special names are displayed". However, visiblename's private name qualifier seems to be (name.startswith('_')) while Python's private name qualifier is something more along the lines of ((name.startswith('__') and ((name[-1] != '_') or ((name[-1] == '_') and (name[-2] != '_')))). Having said that, it would be useful if a command line switch enabled documentation for private names. This would be helpful in a development environment... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189811&group_id=5470 From noreply at sourceforge.net Mon Apr 25 23:06:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Apr 25 23:06:37 2005 Subject: [ python-Bugs-1189819 ] "Atuple containing default argument values ..." Message-ID: Bugs item #1189819, was opened at 2005-04-25 17:06 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189819&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Chad Whitacre (whit537) Assigned to: Nobody/Anonymous (nobody) Summary: "Atuple containing default argument values ..." Initial Comment: should be "A tuple ..." http://python.org/doc/2.4.1/ref/types.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189819&group_id=5470 From noreply at sourceforge.net Tue Apr 26 05:49:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 05:49:25 2005 Subject: [ python-Bugs-1166714 ] [ast branch] fatal error when compiling test_bool.py Message-ID: Bugs item #1166714, was opened at 2005-03-19 16:14 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 Category: Parser/Compiler Group: AST >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: John Ehresman (jpe) Assigned to: Brett Cannon (bcannon) Summary: [ast branch] fatal error when compiling test_bool.py Initial Comment: When using the ast-branch compiler, python exits with a fatal error when compiling test_bool.py or the symplified test case that I'll attach. The immediate problem is that when the compiler is in make_closure after compiling the lambda, self is found to be a LOCAL reference type and make_closure assumes it's a FREE reference if it's not a CELL reference in the enclosing scope. I don't know if self is supposed to be a LOCAL reference or if make_closure should handle LOCAL references. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-25 20:49 Message: Logged In: YES user_id=357491 OK, closed since John verified the version I checked in was the right way to go. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-24 19:28 Message: Logged In: YES user_id=357491 OK, added a comment in bug #1186345 to this fact. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-04-24 18:18 Message: Logged In: YES user_id=22785 Yes, it should be PyDict_Update(free, newfree) like it is in the current CVS. The unicode failures should be fixed by compiling and using tokenizer.c rather than tokenizer_pgen.c, which is mentioned in the comment for patch 1170272 but didn't get changed when the patch was applied. I'm pretty sure tokenizer_pgen.c is not needed. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-18 22:26 Message: Logged In: YES user_id=357491 John, on line 45 of the patch, which becomes line 580 in Python/symtable.c, you call PyDict_Update() with three arguments. Is 'local' supposed to be an argument, or is 'free' the erroneous one? If I change it so that 'local' is removed then the tests die on test_unicode (although a ton of other tests fail). ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 17:22 Message: Logged In: YES user_id=22785 Forgot to add the patch ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 17:21 Message: Logged In: YES user_id=22785 The attached patch seems to fix this by creating a dictionary for each scope's free vars and only propagating up the ones that aren't satisfied by a binding. With this and the other patches, the test suite runs w/o segfaulting up through test_unicode. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-03-22 13:37 Message: Logged In: YES user_id=22785 The issue seems to be that in symtable_analyze, a free dictionary is created for the entire module and if a name is found to be free in any function, it is not converted from a LOCAL to a CELL is any context seen afterwards. I think the free variables need to be recorded on a function-by-function basis. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-03-22 12:34 Message: Logged In: YES user_id=357491 An short example of the failure:: def outer(a): def first(): a def last(a): pass This leads to a fatal Python error. The issue is having a being a local or parameter (can make 'a' be 'a = 1' and not a parameter and still have it fail), the first closure reference it, and then the second one use it as a parameter. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1166714&group_id=5470 From noreply at sourceforge.net Tue Apr 26 05:54:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 05:54:21 2005 Subject: [ python-Bugs-1186345 ] [AST] assert failure on ``eval("u'\Ufffffffe'")`` Message-ID: Bugs item #1186345, was opened at 2005-04-19 16:24 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 Category: Parser/Compiler Group: AST >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] assert failure on ``eval("u'\Ufffffffe'")`` Initial Comment: Isolated the failure of test_unicode to be because of the test of ``eval("u'\Ufffffffe'")``. What is odd is that the Unicode string works fine as a literal at the intepreter prompt. Somehow eval() is triggering this problem. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-25 20:54 Message: Logged In: YES user_id=357491 Fixed by patch #1189210. ---------------------------------------------------------------------- Comment By: John Ehresman (jpe) Date: 2005-04-24 19:35 Message: Logged In: YES user_id=22785 Also look at patch 1189210, which probably should have just been attached to this bug. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2005-04-24 19:27 Message: Logged In: YES user_id=357491 jpe suggests looking at the comment of tracker #1170272; use tokenizer.c instead of tokenizer_pgen.c (this is mentioned in bug #1166714). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1186345&group_id=5470 From noreply at sourceforge.net Tue Apr 26 07:19:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 07:19:43 2005 Subject: [ python-Bugs-1189819 ] "Atuple containing default argument values ..." Message-ID: Bugs item #1189819, was opened at 2005-04-25 16:06 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189819&group_id=5470 Category: Documentation Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Chad Whitacre (whit537) Assigned to: Nobody/Anonymous (nobody) Summary: "Atuple containing default argument values ..." Initial Comment: should be "A tuple ..." http://python.org/doc/2.4.1/ref/types.html ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-26 00:19 Message: Logged In: YES user_id=80475 Okay. Fixed. Thanks. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189819&group_id=5470 From noreply at sourceforge.net Tue Apr 26 08:07:40 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 08:07:43 2005 Subject: [ python-Bugs-1190010 ] [AST] assignment to None allowed Message-ID: Bugs item #1190010, was opened at 2005-04-25 23:07 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190010&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] assignment to None allowed Initial Comment: ``None = 42`` should raise a SyntaxError. Detected by test_compile. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190010&group_id=5470 From noreply at sourceforge.net Tue Apr 26 08:10:10 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 08:10:15 2005 Subject: [ python-Bugs-1190011 ] [AST] distinct code objects not created Message-ID: Bugs item #1190011, was opened at 2005-04-25 23:10 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190011&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] distinct code objects not created Initial Comment: >> def f(): return ((lambda x=1: x), (lambda x=2: x)) >> f1, f2 = f() >> id(f1.func_code) != id(f2.func_code) The above does not hold true. It should according to test_compile (reported in HEAD as bug #1048870). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190011&group_id=5470 From noreply at sourceforge.net Tue Apr 26 08:11:36 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 08:11:38 2005 Subject: [ python-Bugs-1190012 ] ``from sys import stdin, `` doesn't raise a SyntaxError Message-ID: Bugs item #1190012, was opened at 2005-04-25 23:11 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190012&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: ``from sys import stdin,`` doesn't raise a SyntaxError Initial Comment: Python 2.4 raises a SyntaxError: "trailing comma not allowed without surrounding parentheses". Detected by test_compile. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190012&group_id=5470 From noreply at sourceforge.net Tue Apr 26 08:59:50 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 08:59:52 2005 Subject: [ python-Feature Requests-1190033 ] The array module and the buffer interface Message-ID: Feature Requests item #1190033, was opened at 2005-04-25 23:59 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Josiah Carlson (josiahcarlson) Assigned to: Nobody/Anonymous (nobody) Summary: The array module and the buffer interface Initial Comment: It would be terribly convenient if array objects were willing to take any object supporting the buffer interface as initialization or extension via a.fromstring(). They currently offer the buffer interface for other objects to read/write once the array has been created, but they do not accept buffers during creation or extension (except for the typecode 'c'). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 From noreply at sourceforge.net Tue Apr 26 14:26:13 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 14:26:15 2005 Subject: [ python-Bugs-1190204 ] 3.29 site is confusing re site-packages on Windows Message-ID: Bugs item #1190204, was opened at 2005-04-26 12:26 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190204&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Kent Johnson (kjohnson) Assigned to: Nobody/Anonymous (nobody) Summary: 3.29 site is confusing re site-packages on Windows Initial Comment: Library Reference 3.29 site seems to say that site-packages is only searched on Unix and Mac. The relevant sentence is in the third paragraph: "For the tail part, it uses the empty string (on Windows) or it uses first lib/python2.4/site-packages and then lib/site-python (on Unixand Macintosh)." A clearer and more accurate wording would be "For the tail part, it uses the empty string and lib/site-python (on Windows) or it uses first lib/python2.4/site-packages and then lib/site-python (on Unixand Macintosh)." The relevant code is line 187 in site.py. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190204&group_id=5470 From noreply at sourceforge.net Tue Apr 26 19:40:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 19:40:34 2005 Subject: [ python-Bugs-1190451 ] 6.9 First sentence is confusing Message-ID: Bugs item #1190451, was opened at 2005-04-26 19:40 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190451&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nicolas Grilly (ngrilly) Assigned to: Nobody/Anonymous (nobody) Summary: 6.9 First sentence is confusing Initial Comment: The first sentence of section 6.9 "The raise statement" in Python Reference Manual is confusing: "If no expressions are present, raise re-raises the last ***expression*** that was active in the current scope." I guess the author means "the last ***exception*** that was active in the current scope". This corresponds to the behavior of Python and also to following sentence in the manual: "If no ***exception*** is active in the current scope, an exception is raised indicating this error." Therefore, should we replace "expression" by "exception" in the first sentence? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190451&group_id=5470 From noreply at sourceforge.net Tue Apr 26 22:48:54 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 22:48:57 2005 Subject: [ python-Bugs-1190563 ] os.waitpid docs don't specify return value for WNOHANG Message-ID: Bugs item #1190563, was opened at 2005-04-26 16:48 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190563&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: jls (apjenkins) Assigned to: Nobody/Anonymous (nobody) Summary: os.waitpid docs don't specify return value for WNOHANG Initial Comment: The library documentation for os.waitpid() does not specify what the return value should be if the os.WNOHANG flag is given, and the process still hasn't exited. p, v = os.waitpid(pid, os.WNOHANG) Through trial and error I discovered that in this case, os.waitpid returns the tuple (0, 0) if pid is still running. This should be in the documentation for os.waitpid(). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190563&group_id=5470 From noreply at sourceforge.net Tue Apr 26 23:32:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 23:33:01 2005 Subject: [ python-Bugs-1190580 ] SimpleHTTPServer sends wrong c-length and locks up client Message-ID: Bugs item #1190580, was opened at 2005-04-26 23:32 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190580&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Alexander Schremmer (alexanderweb) Assigned to: Nobody/Anonymous (nobody) Summary: SimpleHTTPServer sends wrong c-length and locks up client Initial Comment: On windows, SimpleHTTPServer shows a severe bug: it sends a wrong content-length header. In fact, it sends the wrong data. It differentiates between the mime types "text/" and the remaining ones and decides upon that if it should open the file in binary mode. That is illegal according to the RFCs, any text/ mimetype has to be served using \r\n line endings, therefore it may not read it in non-binary mode. My suggestion: remove the disambiguation (that means the if block). Umm, yeah, my initial reason to report this: if the content-length value is greater than the actual delivered data size, gateways/clients may lock up if they use pipelining. (And they do, BTDT). The reason for that wrong value: the filesize if determined before line end conversion (implicit because of non-binary read file mode) . ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190580&group_id=5470 From noreply at sourceforge.net Tue Apr 26 23:35:22 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 23:35:34 2005 Subject: [ python-Bugs-1188637 ] Python 2.4 Not Recognized by Any Programs Message-ID: Bugs item #1188637, was opened at 2005-04-23 12:45 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188637&group_id=5470 Category: Installation Group: Python 2.4 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Yoshi Nagasaki (gr1ever) Assigned to: Nobody/Anonymous (nobody) Summary: Python 2.4 Not Recognized by Any Programs Initial Comment: Hi, I've tried installing Python 2.4 on two different Windows machines, one running 98 and the other running XP Pro. In both cases, after installing it, none of the programs that use python recognize that it has been installed. These programs include WinCVS and Blender and perhaps others. Only installations of older versions of Python are recognized by the programs (I think I'm using 2.3). Any suggestions on what might be the problem? ---------------------------------------------------------------------- >Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-26 17:35 Message: Logged In: YES user_id=593130 Installation/usage questions belong on comp.lang.python (or gmane.comp.python.general) or the corresponding mailing list at www.python.org. Issues like this have been discussed there by people who know more than I. Do mention whether you used the 2.4 installer or the newer 2.4.1 installer in case there were any changes. Closing as not a Python bug and most likely not an installer bug. If you find out differently, we can reopen. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1188637&group_id=5470 From noreply at sourceforge.net Tue Apr 26 23:46:32 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 23:46:36 2005 Subject: [ python-Bugs-1178872 ] Variable.__init__ uses self.set(), blocking specialization Message-ID: Bugs item #1178872, was opened at 2005-04-07 17:20 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178872&group_id=5470 Category: Tkinter Group: Python 2.4 >Status: Deleted >Resolution: Duplicate Priority: 5 Submitted By: Emil (droun) Assigned to: Martin v. L?wis (loewis) Summary: Variable.__init__ uses self.set(), blocking specialization Initial Comment: in class Variable the constructor uses self.set() to set the value of the variable. This makes it hard and in some cases impossible to make a speciallization of Variable that performs some operation after changing the value. It would be preferable to use Tkinter.Variable.set(self, self._default) example: class ViewChanger(Tkinter.StringVar) : def __init__(self, views) : self.views = views Tkinter.StringVar.__init__(self) def set(self, newview) : self.views.activate(newview) Tkinter.StringVar.set(newview) get() and everything else will use the normal StringVar ipementation, but when the value is changed, a new view is activated. At creation here the self.views varaible will be accessed, method activate() will be called, and it might not even be fully initiallized yet. This is the normal case if you want to supply a variable to a widget, where a variable change may update the widget. ---------------------------------------------------------------------- >Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-26 17:46 Message: Logged In: YES user_id=593130 Duplicate of 1178863 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1178872&group_id=5470 From noreply at sourceforge.net Tue Apr 26 23:54:11 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 23:54:14 2005 Subject: [ python-Bugs-415492 ] Compiler generates relative filenames Message-ID: Bugs item #415492, was opened at 2001-04-11 14:52 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=415492&group_id=5470 Category: Parser/Compiler Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Paul Prescod (prescod) Assigned to: Nobody/Anonymous (nobody) Summary: Compiler generates relative filenames Initial Comment: Make a file called "test.py" ---- import pack print pack.func.func_code.co_filename ----- Make a directory called "pack". Put a file in it called "__init__.py" with the contents: def func(): pass Now run test.py. It will compile a relative path into these modules. Now you can change to any directory on the system and run test.py and it will return the original relative path. The problem is that the relative path is compiled into the .pyc. It should be an absolute path. ---------------------------------------------------------------------- >Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-26 17:54 Message: Logged In: YES user_id=593130 Closing as fixed, on the basis of isandlers report, until someone verifies that it is a current problem. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-23 00:02 Message: Logged In: YES user_id=971153 I tried to reproduce the problem with python 2.4 and I'm getting the absolute path.... So, I guess, the bug can be closed ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2002-04-25 17:08 Message: Logged In: YES user_id=86307 I just ran into this today when trying to find out why pydoc wasn't working for a module of mine: inspect.getmodule calls os.abspath with the filename found in the code object of global functions. If the cwd is different than when the code object was compiled, inspect.getmodule fails. Anyway, it looks to me like most of these relative paths could be caught in the find_module function (of import.c) if, when given an empty string for a path (while iterating over sys.path), find_module called getcwd() and used that instead of the empty string. Of course, this assumes that (on all platforms) opening a bare filename means open the file with that name in the current directory (is that a valid assumption?). Also, it appears from posixmodule.c that getcwd may not always be available. Does this sound like a reasonable idea? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 14:10 Message: Logged In: YES user_id=6380 Why on earth was this assigned to Paul? He's not going to make progress. Assigning back to nobody. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-04-13 17:15 Message: Logged In: YES user_id=31392 Just to clarify. The compiler in question is the builtin compiler. It generates absolute path names unless the .py file is in the current working directory. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-04-13 10:12 Message: Logged In: YES user_id=31392 I'm not clear on what the rules for co_filename are, but it looks like the absolute path is only used for package imports. If you cd into the package directory, run python -c "import __init__", and then later import the package the path name is relative. The compiler package isn't connected to the import mechanism, so there's no way for it to know whether it is compiling a module or a package. I don't think there's a way to do anything better. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=415492&group_id=5470 From noreply at sourceforge.net Tue Apr 26 23:58:06 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Apr 26 23:58:09 2005 Subject: [ python-Bugs-751612 ] smtplib crashes Windows Kernal. Message-ID: Bugs item #751612, was opened at 2003-06-09 18:44 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=751612&group_id=5470 Category: Windows Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: mark lee smith (netytan) Assigned to: Nobody/Anonymous (nobody) Summary: smtplib crashes Windows Kernal. Initial Comment: When trying to use the smtplib module (or modules which use smtplib i.e. email) the Windows Kenal crashes without sending the email. I'm trying to send the email through an external mail server but I imagine the problem would still exist if I was using a local mailserver. ---------------------------------------------------------------------- >Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-26 17:58 Message: Logged In: YES user_id=593130 Seems to be WinME bug. Closing. ---------------------------------------------------------------------- Comment By: mark lee smith (netytan) Date: 2003-06-25 16:20 Message: Logged In: YES user_id=797196 If you mean the error message "..caused an error in..Windows Kernal", you can click ok and it closes most of the time, other times the whole machine bails. I've tested it now on Windows XP pro and home and it sends straight through. I'm gonna try get hold of a 98 box so I can test it but it seems to be just my ME box at the moment. Mark ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-25 10:30 Message: Logged In: YES user_id=80475 Can this be closed? ---------------------------------------------------------------------- Comment By: mark lee smith (netytan) Date: 2003-06-11 18:36 Message: Logged In: YES user_id=797196 Will do, I'll try run the script from my box upstairs and one at work. Thanks for all your help, at least it's sending email, be it you need to send "^D" in that example but with a little reworking it should be a problem. Thank's again. P.S let's blame Microsoft, never did like them, so dodgy lol. If I could get away from Windows I would but unfortunatly it's a commercial standard.. Ta ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2003-06-11 17:36 Message: Logged In: YES user_id=31435 I don't hold out much hope that this will get resolved. Windows has tons of bugs (a lot more than Python!), and it's not uncommon to find a program (Python-based or otherwise) that fails on only one specific Windows box in the world. For example, you could have a corrupted bit in any of the thousand Windows DLLs on your box, or you may simply be behind in running Windows Update. Other possibilities include that you're running a virus checker (those often interfere with normal program operations), or are behind an improperly written firewall. So long as your report is the only one of this nature, there's really no hope for resolving it short you digging into it. I don't see any evidence of a bug in Python here (worse, a crash in a core OS DLL is prima facia evidence of a bug in the OS!). If you can, try running the same thing on a friend's Windows box. I bet you won't have any problem there. ---------------------------------------------------------------------- Comment By: mark lee smith (netytan) Date: 2003-06-11 17:22 Message: Logged In: YES user_id=797196 Ok, I ran your file and it returned the same as when I ran mine, both crashed the Kernal. I think the reason that the mail wasn't being sent before was that I forgot to finish the message with "^D" although the main problem remains, what is the Kernal crashing out :S.. ---------------------------------------------------------------------- Comment By: mark lee smith (netytan) Date: 2003-06-11 16:53 Message: Logged In: YES user_id=797196 Hi, I've replicated what had before exactly, the win32 Kernal still crashes but the mail is delivered :S, don't ask me why it sends now and didn't before, nothing has changed to my knowlage.. Anyway if I've done this right it should be attached to this. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2003-06-09 19:47 Message: Logged In: YES user_id=31435 Just attaching tcap.txt. Note that on Windows, you have to end the msg with Ctrl-Z (not the Unixish Ctrl-D in the example comments). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2003-06-09 19:45 Message: Logged In: YES user_id=31435 I'll show you what I'm looking for: attached is the exact code I just ran, in smtpex.py, and a screen dump from running it, in tcap.txt. I changed the example to use comcast.net's SMTP server. It worked fine for me on Win98SE. I was using Python 2.2.3, and don't know whether that makes a difference. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2003-06-09 19:39 Message: Logged In: YES user_id=31435 I don't understand. If you're not running an SMTP server on localhost, I expect that example to die with socket.error: (10061, 'Connection refused') And indeed, that's what it does when I try it on Win98SE. So I conclude you must have changed the example in some way. If so, the solution to your problem will be found in the details of you changed it. Nobody can guess that, so please attach the code you actually ran. ---------------------------------------------------------------------- Comment By: mark lee smith (netytan) Date: 2003-06-09 19:25 Message: Logged In: YES user_id=797196 Im using Python 2.2.2 and the example in smtplib documentation originally caused the error. It crashes the Kernal on Windows ME and 98, failing to send on Windows XP. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2003-06-09 18:51 Message: Logged In: YES user_id=31435 We're going to need more info, and potentially a lot more. Start with which version of Python and which flavor of Windows. Lots of people use smtplib on all flavors of Windows, and there are no other reports like yours, so it's going to turn out to be something very specific to what you're doing. Also. of possible, attach an executable test program that reliably triggers the problem for you. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=751612&group_id=5470 From noreply at sourceforge.net Wed Apr 27 00:05:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 00:05:28 2005 Subject: [ python-Bugs-755080 ] AssertionError from urllib.retrieve / httplib Message-ID: Bugs item #755080, was opened at 2003-06-15 20:37 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=755080&group_id=5470 Category: Python Library Group: Python 2.3 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Stuart Bishop (zenzen) Assigned to: Nobody/Anonymous (nobody) Summary: AssertionError from urllib.retrieve / httplib Initial Comment: The following statement is occasionally generating AssertionErrors: current_page = urllib.urlopen(action,data).read() Traceback (most recent call last): File "/Users/zen/bin/autospamrep.py", line 161, in ? current_page = handle_spamcop_page(current_page) File "/Users/zen/bin/autospamrep.py", line 137, in handle_spamcop_page current_page = urllib.urlopen(action,data).read() File "/sw/lib/python2.3/httplib.py", line 1150, in read assert not self._line_consumed and self._line_left Fix may be to do the following in LineAndFileWrapper.__init__ (last two lines are new): def __init__(self, line, file): self._line = line self._file = file self._line_consumed = 0 self._line_offset = 0 self._line_left = len(line) if not self._line_left: self._done() ---------------------------------------------------------------------- >Comment By: Terry J. Reedy (tjreedy) Date: 2005-04-26 18:05 Message: Logged In: YES user_id=593130 Closing since this appears to have been fixed in 2.3. If I am mistaken, reopen. ---------------------------------------------------------------------- Comment By: Jon Moses (jmoses) Date: 2003-09-23 08:35 Message: Logged In: YES user_id=55110 I switched from using urllib.urlretrieve / urllib.urlopen to using httplib, since I can debug with it. I no longer get the error this bug is about. The other problem I seemed to be having was related to the data I was recieving, which was generated in part from the data I was passing to the server. I changed the data I was sending (changed ' ' to '%20') and ever thing works fine. Even using urllib.urlopen(). Sorry for the confusion. The data that the server was sending back to the broken request was outputted like this, using httplib.http.set_debuglevel(1): ------start Getting: doi.crossref.org connect: (doi.crossref.org, 80) send: 'GET /servlet/query?usr=<deleted>&pwd=<deleted>&qdata=|Canadian Journal of Fisheries and Aquatic Sciences|Adkison|52||2762||full_text|1|<snip> HTTP/1.0\r\n\r\n' reply: '\n' |Canadian -----------end I don't know if that helps, but maybe. Thanks much. ---------------------------------------------------------------------- Comment By: Jon Moses (jmoses) Date: 2003-09-23 07:52 Message: Logged In: YES user_id=55110 Whups, my bad, I just assumed (and we know what happens then) that this was for python 2.2, since that's what I was having the problem with. My next step was to try with Python 2.3. I'll let you know if it works (since it sounds like it should). And yes, that's what I meant. Data from the http read was still being outputted to the screen, while other output from _past_ where the read was occuring was also being output. I'd end up with output like this: [data from http read] [data from after] [data from http read] and the data was from the same connection. Hopefully the switch to 2.3 makes my issues moot. Thanks ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2003-09-22 23:09 Message: Logged In: YES user_id=31392 jmoses: Are you seeing this problem with Python 2.3? I thought we had fixed the problem in the original report. Also, I'm not sure what you mean by program execution continuing. Do you mean that the for loop finished and the rest of the program continued executing, even though there was data left to read? What would probably help most is a trace of the session with httplib's set_debuglevel() enabled. If that's got sensitive data, you can email it to me privately. ---------------------------------------------------------------------- Comment By: Jon Moses (jmoses) Date: 2003-09-22 16:38 Message: Logged In: YES user_id=55110 I also experience this problem, and it's repeatable. When trying to talk with CrossRef (www.crossref.com) server, I get this same error. I don't know why. All the crossref server does is spit back text. It normally takes between 10 and 20 seconds to recieve all the data. I've successfully viewed the results with mozilla, and with wget. I'd post the URL i'm hitting, but it's a for-pay service. This is the code I'm using: ... ( name, headers ) = urllib.urlretrieve( url ) ... While attempting to recieve this data, I tried doing a: ... u = urllib.urlopen( url ) for line in u.readlines(): print line ... but program execution seemed to continue while the data was being received, which is not cool. I'm not sure if that's expected behaviour or not. Let me know if I can provide you with any more information. -jon ---------------------------------------------------------------------- Comment By: Stuart Bishop (zenzen) Date: 2003-06-24 08:46 Message: Logged In: YES user_id=46639 I've been unable to repeat the problem through a tcpwatch.py proxy, so I'm guessing the trigger is connecting to a fairly loaded server over a 56k modem - possibly the socket is in a bad state and nothing noticed? I'll try not going through tcpwatch.py for a bit and see if I can still trigger the problem in case there was a server problem triggering it that has been fixed. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2003-06-16 15:40 Message: Logged In: YES user_id=31392 Can you reproduce this problem easily? We've seen something like it before, but have had trouble figuring out what goes wrong. ---------------------------------------------------------------------- Comment By: Stuart Bishop (zenzen) Date: 2003-06-15 20:55 Message: Logged In: YES user_id=46639 My suggested fix is wrong. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=755080&group_id=5470 From noreply at sourceforge.net Wed Apr 27 00:10:20 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 00:10:23 2005 Subject: [ python-Bugs-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Bugs item #1190596, was opened at 2005-04-26 18:10 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 From noreply at sourceforge.net Wed Apr 27 00:20:22 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 00:20:25 2005 Subject: [ python-Bugs-1190599 ] dir() docs show incorrect output Message-ID: Bugs item #1190599, was opened at 2005-04-26 16:20 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190599&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Martin Chase (stillflame) Assigned to: Nobody/Anonymous (nobody) Summary: dir() docs show incorrect output Initial Comment: on the web at http://docs.python.org/tut/node8.html under "6.3 The dir() Function", the following text reports incorrectly what the dir() call would return: >>> a = [1, 2, 3, 4, 5] >>> import fibo, sys >>> fib = fibo.fib >>> dir() ['__builtins__', '__doc__', '__file__', '__name__', 'fib', 'fib2'] it should have 'a' and 'fibo', but not 'fib2'. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190599&group_id=5470 From noreply at sourceforge.net Wed Apr 27 03:19:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 03:19:06 2005 Subject: [ python-Feature Requests-1190689 ] logging module '.' behavior Message-ID: Feature Requests item #1190689, was opened at 2005-04-26 20:19 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: logging module '.' behavior Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Wed Apr 27 03:35:57 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 03:35:59 2005 Subject: [ python-Feature Requests-1190701 ] Add 'before' and 'after' methods to Strings Message-ID: Feature Requests item #1190701, was opened at 2005-04-26 20:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: Add 'before' and 'after' methods to Strings Initial Comment: GNU String used to have two very useful methods, 'before' and 'after'. These are so useful I keep them defined in an __init__.py file. (Unfortunately, I do not know how to make them methods, instead of global functions.) Usage: >>> "root.sub".before(".") 'root' >>> "root.sub1.sub2".after("root.sub1") '.sub2' They work like s.split(word)[0], and s.split(word)[-1], but they are so intuitive they ought to be part of the interface. I'm not sure whether they should raise exceptions on failure, or simply return the whole string. -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 From noreply at sourceforge.net Wed Apr 27 05:36:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 05:36:18 2005 Subject: [ python-Bugs-1190599 ] dir() docs show incorrect output Message-ID: Bugs item #1190599, was opened at 2005-04-27 07:20 Message generated for change (Comment added) made by quiver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190599&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Martin Chase (stillflame) Assigned to: Nobody/Anonymous (nobody) Summary: dir() docs show incorrect output Initial Comment: on the web at http://docs.python.org/tut/node8.html under "6.3 The dir() Function", the following text reports incorrectly what the dir() call would return: >>> a = [1, 2, 3, 4, 5] >>> import fibo, sys >>> fib = fibo.fib >>> dir() ['__builtins__', '__doc__', '__file__', '__name__', 'fib', 'fib2'] it should have 'a' and 'fibo', but not 'fib2'. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2005-04-27 12:36 Message: Logged In: YES user_id=671362 A minor nit. > it should have 'a' and 'fibo', 'sys' is also missing from the list. >>> dir() ['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib', 'fibo', ' sys'] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190599&group_id=5470 From noreply at sourceforge.net Wed Apr 27 05:58:19 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 05:58:22 2005 Subject: [ python-Bugs-412436 ] compileall doesn't notice syntax errors Message-ID: Bugs item #412436, was opened at 2001-03-30 02:59 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=412436&group_id=5470 Category: Python Library Group: None Status: Open Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: compileall doesn't notice syntax errors Initial Comment: compileall.py returns an exit code to indicate the success or failure of compilation. This feature was added in compileall.py revision 1.7 in response to distutils message http://mail.python.org/pipermail/distutils-sig/1999-March/000201.html This is not as useful as it looks because a prior change to py_compile.py (revision 1.13) catches syntax errors, hiding them completely from compileall.py, so compileall.py can't report the failure to its caller. ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2005-04-26 20:58 Message: Logged In: YES user_id=971153 I did a small test ~>echo '"garab' > garbage.py ~>python -m compileall . and got expected: SyntaxError: EOL while scanning single-quoted string and exit status of 1 So the problem is fixed in latest python and bug can be closed ---------------------------------------------------------------------- Comment By: Just van Rossum (jvr) Date: 2003-01-05 13:01 Message: Logged In: YES user_id=92689 The change to compileall.py has been in CVS for a long time. The strange thing is that the accompanying py_compile.py patch wasn't checked in. All I did was update that change for current CVS. Jeremy? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-01-05 12:25 Message: Logged In: YES user_id=33168 I definitely think Just's change to py_compile.py (return 0 or 1 from compile()) should be checked in. This doesn't hurt and is backwards compatible. I think a NEWS entry and doc updates (Doc/lib/libpycompile.tex) should be done as well. The change to compileall.py is also probably ok, but I didn't look as closely (the patch failed to apply for me in 2.3). ---------------------------------------------------------------------- Comment By: Just van Rossum (jvr) Date: 2003-01-03 04:29 Message: Logged In: YES user_id=92689 Reopening this bug as it's entirely unclear why the py_compile.py change wasn't checked in. This has resulted in a new bug, #653301. I've attached a patch that's updated for current CVS (rev. 1.23 of py_compile.py). Since py_compile.py is crucial to the install process I'd rather have someone else make the decision to check this in or not. ---------------------------------------------------------------------- Comment By: Peter Maxwell (pm67nz) Date: 2001-09-09 20:31 Message: Logged In: YES user_id=320286 I don't see how this can be fixed in compileall.py since the problem is with py_compile.py. py_compile.compile (up to and including the latest version I see in CVS, revision 1.18) only ever returns None, so the code in compileall.py revision 1.19 that says: ok = py_compile.compile(fullname, None, dfile) and if ok == 0: can't possibly work. Caveat: this comment based on reading code, not running it. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-04-17 18:23 Message: Logged In: YES user_id=31392 Fixed in rev 1.9 of compileall.py. Note that this fix causes a bunch of changes to the test suite, so that files containing syntaxerrors are not compiled by compileall. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-04-13 14:16 Message: Logged In: YES user_id=31392 Will fix following the 2.1 release ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-04-13 07:31 Message: Logged In: YES user_id=31392 I think this is easy enough to fix, but I don't know what unintended side-effects the fix will have. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=412436&group_id=5470 From noreply at sourceforge.net Wed Apr 27 12:00:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 12:00:40 2005 Subject: [ python-Bugs-1190599 ] dir() docs show incorrect output Message-ID: Bugs item #1190599, was opened at 2005-04-26 23:20 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190599&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Martin Chase (stillflame) >Assigned to: Michael Hudson (mwh) Summary: dir() docs show incorrect output Initial Comment: on the web at http://docs.python.org/tut/node8.html under "6.3 The dir() Function", the following text reports incorrectly what the dir() call would return: >>> a = [1, 2, 3, 4, 5] >>> import fibo, sys >>> fib = fibo.fib >>> dir() ['__builtins__', '__doc__', '__file__', '__name__', 'fib', 'fib2'] it should have 'a' and 'fibo', but not 'fib2'. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-27 11:00 Message: Logged In: YES user_id=6656 Fixed in rev 1.270 of Doc/tut.tex. Thanks for the report! ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2005-04-27 04:36 Message: Logged In: YES user_id=671362 A minor nit. > it should have 'a' and 'fibo', 'sys' is also missing from the list. >>> dir() ['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib', 'fibo', ' sys'] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190599&group_id=5470 From noreply at sourceforge.net Wed Apr 27 14:23:00 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 14:23:04 2005 Subject: [ python-Bugs-640110 ] email.Header misparses mixed headers Message-ID: Bugs item #640110, was opened at 2002-11-18 15:33 Message generated for change (Comment added) made by kalinda You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640110&group_id=5470 Category: Python Library Group: Python 2.2.2 Status: Closed Resolution: Fixed Priority: 5 Submitted By: Anders Hammarquist (iko) Assigned to: Barry A. Warsaw (bwarsaw) Summary: email.Header misparses mixed headers Initial Comment: email.Header.decode_header() misparses headers with both encoded an unencoded words. This example from RFC2047 =?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be> gets parsed as Andr?Pirard <PIRARD@vm1.ulg.ac.be> where there should obviously be a space between Andr? and Pirard. RFC2047 says to ignore spaces between encoded words (but not between encoded and unencoded words, though it doesn't explicitly say so from what I could find, and obviously not between unencoded words). Also, I see it's trying to handle continuation lines, but it only does it if there are encoded words in the continuation line. It barfs badly on this test case: 'Re: =?mac-iceland?q?r=8Aksm=9Arg=8Cs?= baz\n foo bar =?mac-iceland?q?r=8Aksm=9Arg=8Cs?=' I think I'll just do a patch... /Anders P.S. It seems at least remotely related to Bug#552957 ---------------------------------------------------------------------- Comment By: jonny reichwald (kalinda) Date: 2005-04-27 14:23 Message: Logged In: YES user_id=661399 I am using python 2.4 and still have this problem. To be more exact, line 73 in Header.py still strips the parts. Is there a reason for this not being fixed? ---------------------------------------------------------------------- Comment By: Anders Hammarquist (iko) Date: 2003-03-06 17:43 Message: Logged In: YES user_id=14 Looks OK. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2003-03-06 17:21 Message: Logged In: YES user_id=12800 Try current cvs. ---------------------------------------------------------------------- Comment By: Anders Hammarquist (iko) Date: 2003-03-06 15:15 Message: Logged In: YES user_id=14 The first bug is still there... With version 1.19 from CVS I get this with my example: >>> print unicode(Header.make_header(Header.decode_header('=?ISO-8859-1?Q?Andr=E9?= Pirard <PIRARD@vm1.ulg.ac.be>'))).encode('latin-1') Andr?Pirard <PIRARD@vm1.ulg.ac.be> (The problem is that whitespaces get stripped of on line 91: unenc = parts.pop(0).strip() before we know whether they are significant or not. The continuation line bug seems to be fixed however. /Anders ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2003-03-06 07:50 Message: Logged In: YES user_id=12800 The first bug above has already been fixed in email 2.5 (python 2.3 cvs). The second pointed to a real bug, now fixed I believe. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=640110&group_id=5470 From noreply at sourceforge.net Wed Apr 27 16:34:37 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 16:34:40 2005 Subject: [ python-Bugs-1191043 ] bz2 RuntimeError when decompressing file Message-ID: Bugs item #1191043, was opened at 2005-04-27 10:34 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris AtLee (catlee) Assigned to: Nobody/Anonymous (nobody) Summary: bz2 RuntimeError when decompressing file Initial Comment: The following code: echo -n Testing123 | bzip2 > test.bz2 python -c "import bz2; lines = bz2.BZ2File('test.bz2').readlines()" produces this output: Traceback (most recent call last): File "", line 1, in ? RuntimeError: wrong sequence of bz2 library commands used Tested on Python 2.4.1 (debian unstable - April 1 2005), and Python 2.3.5 (debian unstable - May 26 2005) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 From noreply at sourceforge.net Wed Apr 27 17:55:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 17:55:58 2005 Subject: [ python-Bugs-1191104 ] Warning ``error`` filter action is ignored. Message-ID: Bugs item #1191104, was opened at 2005-04-27 17:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ivan Vilata i Balaguer (ivilata) Assigned to: Nobody/Anonymous (nobody) Summary: Warning ``error`` filter action is ignored. Initial Comment: Hi all. It seems that setting the ``error`` action on a warning message once it has already been triggered does not allow it to be triggered again. As usual, the code is clearer than the explanation:: import warnings def do_warn(): warnings.warn("A warning.", DeprecationWarning) do_warn() warnings.filterwarnings('error', category=DeprecationWarning) do_warn() warnings.filterwarnings('always', category=DeprecationWarning) do_warn() The output of this program is:: nowarn.py:4: DeprecationWarning: A warning. warnings.warn("A warning.", DeprecationWarning) There is no exception raised, though the filter was instructed to turn the warning into an error. Take note that using ``always`` has similar results. Does it work in that way on purpose? I find it quite counterintuitive, IMHO. If this behaviour is intended, what would be the way to turn the second warning into an exception? Thanks! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 From noreply at sourceforge.net Wed Apr 27 22:37:55 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Apr 27 22:37:58 2005 Subject: [ python-Feature Requests-1190689 ] logging module '.' behavior Message-ID: Feature Requests item #1190689, was opened at 2005-04-26 20:19 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: logging module '.' behavior Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-27 15:37 Message: Logged In: YES user_id=1267419 Novices always ask, "Why did it print 'root'? Where did that come from? After discussing this with some other "logging" module users, I think we've come up with a very good idea, which would maintain BACKWARDS COMPATIBILITY. Essentially, treat the logging module as a shell and the logger name as a path. Specifically, * Let the global logging functions operate on the "current worrking logger", which by default is "." (Let "root" be an alias for ".") * Change getLogger() so that it works on both absolute and relative logger paths. (Since the default current logger is "root", we maintain backwards compatibility.) * Change the format function so that %(name)s shows the relative path, if the absolute path starts with the current working logger name. * Add a format keyword, %(absname)s, which prints the absolute logger path. * Add another format keyword, %(logger)s, which prints what most people expect to see: the absolute logger name, sans the leading dot. (The "root" or "." logger would display as "", exactly the way it is usually accessed.) * Add global functions, change_current_logger() and get_current_logger(). * Add global function, alias(). Always create an alias for "root" to "." Examples:: from logging import * log = getLogger() #or getLogger(".") or getLogger("root") h1 = StreamHandler() f1 = Formatter("[%(name)s]%(message)s") h1.setFormatter(f1) log.addHandler(h1) h2 = StreamHandler() f2 = Formatter("[%(absname)s]%(message)s") h2.setFormatter(f2) log.addHandler(h2) h3 = StreamHandler() f3 = Formatter("[%(logger)s]%(message)s") h3.setFormatter(f3) log.addHandler(h3) log.error("First message") # ... child = getLogger("child") # or getLogger(".child") child.error("Bad news") This should print: [root]First message [.]First message []First message [child]Bad news [.child]Bad news [child]Bad news This would create tremendous flexibility, add some clarity to the meaning of the "root" logger, and still maintain complete backwards compatibility. I am willing to make the changes myself, including UnitTests, if there is agreement that they would be adopted. (Note that String.before() and String.after() would make the coding a little easier/clearer, but that's a different feature request.) -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Thu Apr 28 03:15:03 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 03:15:09 2005 Subject: [ python-Feature Requests-1191420 ] cStringIO has reset(), but StringIO does not Message-ID: Feature Requests item #1191420, was opened at 2005-04-27 20:15 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191420&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: cStringIO has reset(), but StringIO does not Initial Comment: from cStringIO import StringIO s = StringIO() s.reset() from StringIO import StringIO s = StringIO() s.reset() # error! reset() is not critical, since it just does the same thing as seek(0), but cStringIO and StringIO are supposed to be interchangeable. -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191420&group_id=5470 From noreply at sourceforge.net Thu Apr 28 05:30:55 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 05:30:57 2005 Subject: [ python-Bugs-1191458 ] [AST] Failing tests Message-ID: Bugs item #1191458, was opened at 2005-04-27 20:30 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191458&group_id=5470 Category: Parser/Compiler Group: AST Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] Failing tests Initial Comment: This tracker item is to be used to keep track of what tests are currently failing on the AST branch. This is somewhat important sinced there are so many failures it can be hard to detect if a change fixed what it was supposed to or actually managed to break more code. When posting follow-ups of fixed tests, please re-list the currently failing tests so as to make it simple to reference the current state of things. So, the current offenders are (from a straight ``regrtest.py`` run on my OS X box, which always has test__locale fail so I removed it):: test_compile test_cookielib test_dis test_doctest test_future test_genexps test_inspect test_itertools test_new test_peepholer test_scope test_socket test_sort test_subprocess test_symtable test_syntax test_trace test_traceback test_warnings test_zipfile ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191458&group_id=5470 From noreply at sourceforge.net Thu Apr 28 05:33:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 05:33:18 2005 Subject: [ python-Bugs-1190010 ] [AST] assignment to None allowed Message-ID: Bugs item #1190010, was opened at 2005-04-25 23:07 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190010&group_id=5470 Category: Parser/Compiler Group: AST >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: [AST] assignment to None allowed Initial Comment: ``None = 42`` should raise a SyntaxError. Detected by test_compile. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-27 20:33 Message: Logged In: YES user_id=357491 Fixed in rev. 1.1.2.62 . One less failure in test_compile. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190010&group_id=5470 From noreply at sourceforge.net Thu Apr 28 06:57:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 06:57:19 2005 Subject: [ python-Feature Requests-1191420 ] cStringIO has reset(), but StringIO does not Message-ID: Feature Requests item #1191420, was opened at 2005-04-27 20:15 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191420&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: cStringIO has reset(), but StringIO does not Initial Comment: from cStringIO import StringIO s = StringIO() s.reset() from StringIO import StringIO s = StringIO() s.reset() # error! reset() is not critical, since it just does the same thing as seek(0), but cStringIO and StringIO are supposed to be interchangeable. -cxdunn ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-27 23:57 Message: Logged In: YES user_id=80475 On the surface, it would seem like a good idea; however, reset() was a mistake that should not be propagated. The API should be as file-like as possible. Since there is no file.reset(), there should not be a StringO.reset(). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191420&group_id=5470 From noreply at sourceforge.net Thu Apr 28 07:15:53 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 07:15:55 2005 Subject: [ python-Feature Requests-1190701 ] Add 'before' and 'after' methods to Strings Message-ID: Feature Requests item #1190701, was opened at 2005-04-26 20:35 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: Add 'before' and 'after' methods to Strings Initial Comment: GNU String used to have two very useful methods, 'before' and 'after'. These are so useful I keep them defined in an __init__.py file. (Unfortunately, I do not know how to make them methods, instead of global functions.) Usage: >>> "root.sub".before(".") 'root' >>> "root.sub1.sub2".after("root.sub1") '.sub2' They work like s.split(word)[0], and s.split(word)[-1], but they are so intuitive they ought to be part of the interface. I'm not sure whether they should raise exceptions on failure, or simply return the whole string. -cxdunn ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 00:15 Message: Logged In: YES user_id=80475 I'm -1 on expanding the string API for something so easily coded with existing primitives: >>> s = "root.sub" >>> t = "." >>> s[:s.find(t)] 'root' >>> s = "root.sub1.sub2" >>> t = "root.sub1" >>> s[s.find(sep)+len(sep):] 'sub1.sub2' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 From noreply at sourceforge.net Thu Apr 28 07:16:16 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 07:16:19 2005 Subject: [ python-Feature Requests-1190689 ] logging module '.' behavior Message-ID: Feature Requests item #1190689, was opened at 2005-04-26 20:19 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) >Assigned to: Vinay Sajip (vsajip) Summary: logging module '.' behavior Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-27 15:37 Message: Logged In: YES user_id=1267419 Novices always ask, "Why did it print 'root'? Where did that come from? After discussing this with some other "logging" module users, I think we've come up with a very good idea, which would maintain BACKWARDS COMPATIBILITY. Essentially, treat the logging module as a shell and the logger name as a path. Specifically, * Let the global logging functions operate on the "current worrking logger", which by default is "." (Let "root" be an alias for ".") * Change getLogger() so that it works on both absolute and relative logger paths. (Since the default current logger is "root", we maintain backwards compatibility.) * Change the format function so that %(name)s shows the relative path, if the absolute path starts with the current working logger name. * Add a format keyword, %(absname)s, which prints the absolute logger path. * Add another format keyword, %(logger)s, which prints what most people expect to see: the absolute logger name, sans the leading dot. (The "root" or "." logger would display as "", exactly the way it is usually accessed.) * Add global functions, change_current_logger() and get_current_logger(). * Add global function, alias(). Always create an alias for "root" to "." Examples:: from logging import * log = getLogger() #or getLogger(".") or getLogger("root") h1 = StreamHandler() f1 = Formatter("[%(name)s]%(message)s") h1.setFormatter(f1) log.addHandler(h1) h2 = StreamHandler() f2 = Formatter("[%(absname)s]%(message)s") h2.setFormatter(f2) log.addHandler(h2) h3 = StreamHandler() f3 = Formatter("[%(logger)s]%(message)s") h3.setFormatter(f3) log.addHandler(h3) log.error("First message") # ... child = getLogger("child") # or getLogger(".child") child.error("Bad news") This should print: [root]First message [.]First message []First message [child]Bad news [.child]Bad news [child]Bad news This would create tremendous flexibility, add some clarity to the meaning of the "root" logger, and still maintain complete backwards compatibility. I am willing to make the changes myself, including UnitTests, if there is agreement that they would be adopted. (Note that String.before() and String.after() would make the coding a little easier/clearer, but that's a different feature request.) -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Thu Apr 28 07:20:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 07:20:38 2005 Subject: [ python-Feature Requests-1190033 ] The array module and the buffer interface Message-ID: Feature Requests item #1190033, was opened at 2005-04-26 01:59 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Josiah Carlson (josiahcarlson) Assigned to: Nobody/Anonymous (nobody) Summary: The array module and the buffer interface Initial Comment: It would be terribly convenient if array objects were willing to take any object supporting the buffer interface as initialization or extension via a.fromstring(). They currently offer the buffer interface for other objects to read/write once the array has been created, but they do not accept buffers during creation or extension (except for the typecode 'c'). ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 00:20 Message: Logged In: YES user_id=80475 Do you have a patch? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 From noreply at sourceforge.net Thu Apr 28 08:37:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 08:37:17 2005 Subject: [ python-Feature Requests-1190033 ] The array module and the buffer interface Message-ID: Feature Requests item #1190033, was opened at 2005-04-25 23:59 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Josiah Carlson (josiahcarlson) Assigned to: Nobody/Anonymous (nobody) Summary: The array module and the buffer interface Initial Comment: It would be terribly convenient if array objects were willing to take any object supporting the buffer interface as initialization or extension via a.fromstring(). They currently offer the buffer interface for other objects to read/write once the array has been created, but they do not accept buffers during creation or extension (except for the typecode 'c'). ---------------------------------------------------------------------- >Comment By: Josiah Carlson (josiahcarlson) Date: 2005-04-27 23:37 Message: Logged In: YES user_id=341410 Not right now, but I could probably have one for you tomorrow. You want doc updates and tests too? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-27 22:20 Message: Logged In: YES user_id=80475 Do you have a patch? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 From noreply at sourceforge.net Thu Apr 28 08:40:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 08:40:33 2005 Subject: [ python-Feature Requests-1190701 ] Add 'before' and 'after' methods to Strings Message-ID: Feature Requests item #1190701, was opened at 2005-04-26 20:35 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: Add 'before' and 'after' methods to Strings Initial Comment: GNU String used to have two very useful methods, 'before' and 'after'. These are so useful I keep them defined in an __init__.py file. (Unfortunately, I do not know how to make them methods, instead of global functions.) Usage: >>> "root.sub".before(".") 'root' >>> "root.sub1.sub2".after("root.sub1") '.sub2' They work like s.split(word)[0], and s.split(word)[-1], but they are so intuitive they ought to be part of the interface. I'm not sure whether they should raise exceptions on failure, or simply return the whole string. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:40 Message: Logged In: YES user_id=1267419 Your examples prove my point: >>> s = "root.sub" >>> t = "fubar" >>> s[:s.find(t)] 'root.su' >>> s = "root.sub1.sub2" >>> t = "fubar" >>> s[s.find(sep)+len(sep):] '.sub1.sub2' string.find() is the wrong way. I can live with string.split(): >>> "root.sub1.sub2" >>> t = '.' >>> s.split(t)[0] 'root' >>> s.split(t)[-1] 'sub2' >>> t = "fubar" >>> s.split(t)[0] 'root.sub1.sub2' >>> s.split(t)[-1] 'root.sub1.sub2' This is not terrible, but the desired behavior is really more like strip/rstrip:: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] It's a question of elegance. These are very useful, infuitive functions, and I cannot add them to string myself. And as you've seen, it's easy to create bugs when you try to do this on the fly. Reconsider? If not, I'll just post it in the Cookbook, to point out the dangers of relying on string.find. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 00:15 Message: Logged In: YES user_id=80475 I'm -1 on expanding the string API for something so easily coded with existing primitives: >>> s = "root.sub" >>> t = "." >>> s[:s.find(t)] 'root' >>> s = "root.sub1.sub2" >>> t = "root.sub1" >>> s[s.find(sep)+len(sep):] 'sub1.sub2' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 From noreply at sourceforge.net Thu Apr 28 08:50:22 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 08:50:25 2005 Subject: [ python-Feature Requests-1190701 ] Add 'before' and 'after' methods to Strings Message-ID: Feature Requests item #1190701, was opened at 2005-04-26 20:35 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: Add 'before' and 'after' methods to Strings Initial Comment: GNU String used to have two very useful methods, 'before' and 'after'. These are so useful I keep them defined in an __init__.py file. (Unfortunately, I do not know how to make them methods, instead of global functions.) Usage: >>> "root.sub".before(".") 'root' >>> "root.sub1.sub2".after("root.sub1") '.sub2' They work like s.split(word)[0], and s.split(word)[-1], but they are so intuitive they ought to be part of the interface. I'm not sure whether they should raise exceptions on failure, or simply return the whole string. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:50 Message: Logged In: YES user_id=1267419 Your examples prove my point:: >>> s = "Monty.Python" >>> t = "fubar" >>> s[:s.find(t)] 'Monty.Pytho' >>> s[s.find(t)+len(t):] 'y.Python' Of course, this would work: >>> s.split(t)[0] 'Monty.Python' >>> s.split(t)[-1] 'Monty.Python' That is not terrible, but the behavior I want is actually more like strip()/rstrip():: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] Besides, it's a question of elegance. These are very useful little functions, which look wonderful as methods of string, and the on-the-fly solutions are prone to error. Reconsider? If not, I'll just post it to the Cookbook (without your name -- I'm not trying to embarrass anyone) to point out the danger of relying on string.find(). -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:40 Message: Logged In: YES user_id=1267419 Your examples prove my point: >>> s = "root.sub" >>> t = "fubar" >>> s[:s.find(t)] 'root.su' >>> s = "root.sub1.sub2" >>> t = "fubar" >>> s[s.find(sep)+len(sep):] '.sub1.sub2' string.find() is the wrong way. I can live with string.split(): >>> "root.sub1.sub2" >>> t = '.' >>> s.split(t)[0] 'root' >>> s.split(t)[-1] 'sub2' >>> t = "fubar" >>> s.split(t)[0] 'root.sub1.sub2' >>> s.split(t)[-1] 'root.sub1.sub2' This is not terrible, but the desired behavior is really more like strip/rstrip:: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] It's a question of elegance. These are very useful, infuitive functions, and I cannot add them to string myself. And as you've seen, it's easy to create bugs when you try to do this on the fly. Reconsider? If not, I'll just post it in the Cookbook, to point out the dangers of relying on string.find. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 00:15 Message: Logged In: YES user_id=80475 I'm -1 on expanding the string API for something so easily coded with existing primitives: >>> s = "root.sub" >>> t = "." >>> s[:s.find(t)] 'root' >>> s = "root.sub1.sub2" >>> t = "root.sub1" >>> s[s.find(sep)+len(sep):] 'sub1.sub2' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 From noreply at sourceforge.net Thu Apr 28 08:54:11 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 08:54:14 2005 Subject: [ python-Feature Requests-1190701 ] Add 'before' and 'after' methods to Strings Message-ID: Feature Requests item #1190701, was opened at 2005-04-26 20:35 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: Add 'before' and 'after' methods to Strings Initial Comment: GNU String used to have two very useful methods, 'before' and 'after'. These are so useful I keep them defined in an __init__.py file. (Unfortunately, I do not know how to make them methods, instead of global functions.) Usage: >>> "root.sub".before(".") 'root' >>> "root.sub1.sub2".after("root.sub1") '.sub2' They work like s.split(word)[0], and s.split(word)[-1], but they are so intuitive they ought to be part of the interface. I'm not sure whether they should raise exceptions on failure, or simply return the whole string. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:54 Message: Logged In: YES user_id=1267419 Sorry for the double-post. I thought I'd lost it and re-typed the whole thing. Read the top one only -- less buggy. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:50 Message: Logged In: YES user_id=1267419 Your examples prove my point:: >>> s = "Monty.Python" >>> t = "fubar" >>> s[:s.find(t)] 'Monty.Pytho' >>> s[s.find(t)+len(t):] 'y.Python' Of course, this would work: >>> s.split(t)[0] 'Monty.Python' >>> s.split(t)[-1] 'Monty.Python' That is not terrible, but the behavior I want is actually more like strip()/rstrip():: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] Besides, it's a question of elegance. These are very useful little functions, which look wonderful as methods of string, and the on-the-fly solutions are prone to error. Reconsider? If not, I'll just post it to the Cookbook (without your name -- I'm not trying to embarrass anyone) to point out the danger of relying on string.find(). -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:40 Message: Logged In: YES user_id=1267419 Your examples prove my point: >>> s = "root.sub" >>> t = "fubar" >>> s[:s.find(t)] 'root.su' >>> s = "root.sub1.sub2" >>> t = "fubar" >>> s[s.find(sep)+len(sep):] '.sub1.sub2' string.find() is the wrong way. I can live with string.split(): >>> "root.sub1.sub2" >>> t = '.' >>> s.split(t)[0] 'root' >>> s.split(t)[-1] 'sub2' >>> t = "fubar" >>> s.split(t)[0] 'root.sub1.sub2' >>> s.split(t)[-1] 'root.sub1.sub2' This is not terrible, but the desired behavior is really more like strip/rstrip:: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] It's a question of elegance. These are very useful, infuitive functions, and I cannot add them to string myself. And as you've seen, it's easy to create bugs when you try to do this on the fly. Reconsider? If not, I'll just post it in the Cookbook, to point out the dangers of relying on string.find. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 00:15 Message: Logged In: YES user_id=80475 I'm -1 on expanding the string API for something so easily coded with existing primitives: >>> s = "root.sub" >>> t = "." >>> s[:s.find(t)] 'root' >>> s = "root.sub1.sub2" >>> t = "root.sub1" >>> s[s.find(sep)+len(sep):] 'sub1.sub2' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 From noreply at sourceforge.net Thu Apr 28 08:58:04 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 08:58:06 2005 Subject: [ python-Feature Requests-1190701 ] Add 'before' and 'after' methods to Strings Message-ID: Feature Requests item #1190701, was opened at 2005-04-26 20:35 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: Add 'before' and 'after' methods to Strings Initial Comment: GNU String used to have two very useful methods, 'before' and 'after'. These are so useful I keep them defined in an __init__.py file. (Unfortunately, I do not know how to make them methods, instead of global functions.) Usage: >>> "root.sub".before(".") 'root' >>> "root.sub1.sub2".after("root.sub1") '.sub2' They work like s.split(word)[0], and s.split(word)[-1], but they are so intuitive they ought to be part of the interface. I'm not sure whether they should raise exceptions on failure, or simply return the whole string. -cxdunn ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 01:58 Message: Logged In: YES user_id=80475 You read too much into a simplified example. Test the find() result for -1 and be done with it. Go ahead with a cookbook recipe if you think that no one else is bright enough to write their own. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:54 Message: Logged In: YES user_id=1267419 Sorry for the double-post. I thought I'd lost it and re-typed the whole thing. Read the top one only -- less buggy. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:50 Message: Logged In: YES user_id=1267419 Your examples prove my point:: >>> s = "Monty.Python" >>> t = "fubar" >>> s[:s.find(t)] 'Monty.Pytho' >>> s[s.find(t)+len(t):] 'y.Python' Of course, this would work: >>> s.split(t)[0] 'Monty.Python' >>> s.split(t)[-1] 'Monty.Python' That is not terrible, but the behavior I want is actually more like strip()/rstrip():: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] Besides, it's a question of elegance. These are very useful little functions, which look wonderful as methods of string, and the on-the-fly solutions are prone to error. Reconsider? If not, I'll just post it to the Cookbook (without your name -- I'm not trying to embarrass anyone) to point out the danger of relying on string.find(). -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:40 Message: Logged In: YES user_id=1267419 Your examples prove my point: >>> s = "root.sub" >>> t = "fubar" >>> s[:s.find(t)] 'root.su' >>> s = "root.sub1.sub2" >>> t = "fubar" >>> s[s.find(sep)+len(sep):] '.sub1.sub2' string.find() is the wrong way. I can live with string.split(): >>> "root.sub1.sub2" >>> t = '.' >>> s.split(t)[0] 'root' >>> s.split(t)[-1] 'sub2' >>> t = "fubar" >>> s.split(t)[0] 'root.sub1.sub2' >>> s.split(t)[-1] 'root.sub1.sub2' This is not terrible, but the desired behavior is really more like strip/rstrip:: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] It's a question of elegance. These are very useful, infuitive functions, and I cannot add them to string myself. And as you've seen, it's easy to create bugs when you try to do this on the fly. Reconsider? If not, I'll just post it in the Cookbook, to point out the dangers of relying on string.find. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 00:15 Message: Logged In: YES user_id=80475 I'm -1 on expanding the string API for something so easily coded with existing primitives: >>> s = "root.sub" >>> t = "." >>> s[:s.find(t)] 'root' >>> s = "root.sub1.sub2" >>> t = "root.sub1" >>> s[s.find(sep)+len(sep):] 'sub1.sub2' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 From noreply at sourceforge.net Thu Apr 28 08:59:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 08:59:52 2005 Subject: [ python-Bugs-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Bugs item #1190596, was opened at 2005-04-26 17:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 01:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 From noreply at sourceforge.net Thu Apr 28 09:06:09 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 09:06:12 2005 Subject: [ python-Bugs-1191043 ] bz2 RuntimeError when decompressing file Message-ID: Bugs item #1191043, was opened at 2005-04-27 09:34 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris AtLee (catlee) Assigned to: Nobody/Anonymous (nobody) Summary: bz2 RuntimeError when decompressing file Initial Comment: The following code: echo -n Testing123 | bzip2 > test.bz2 python -c "import bz2; lines = bz2.BZ2File('test.bz2').readlines()" produces this output: Traceback (most recent call last): File "", line 1, in ? RuntimeError: wrong sequence of bz2 library commands used Tested on Python 2.4.1 (debian unstable - April 1 2005), and Python 2.3.5 (debian unstable - May 26 2005) ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:06 Message: Logged In: YES user_id=80475 The looks like correct behavior to me. The test.bz2 file is not in a valid bz2 format. I suspect that you've misread the BZ2File API which is intended for reading and writing uncompressed data to and from a file in a bz2 format (where the data is stored in compressed form). If this interpretation of the bug report is correct, please mark as not-a-bug and close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 From noreply at sourceforge.net Thu Apr 28 09:14:34 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 09:14:37 2005 Subject: [ python-Bugs-1190451 ] 6.9 First sentence is confusing Message-ID: Bugs item #1190451, was opened at 2005-04-26 12:40 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190451&group_id=5470 Category: Documentation >Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nicolas Grilly (ngrilly) Assigned to: Nobody/Anonymous (nobody) Summary: 6.9 First sentence is confusing Initial Comment: The first sentence of section 6.9 "The raise statement" in Python Reference Manual is confusing: "If no expressions are present, raise re-raises the last ***expression*** that was active in the current scope." I guess the author means "the last ***exception*** that was active in the current scope". This corresponds to the behavior of Python and also to following sentence in the manual: "If no ***exception*** is active in the current scope, an exception is raised indicating this error." Therefore, should we replace "expression" by "exception" in the first sentence? ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:14 Message: Logged In: YES user_id=80475 Fixed. Thanks for the report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190451&group_id=5470 From noreply at sourceforge.net Thu Apr 28 09:21:32 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 09:21:35 2005 Subject: [ python-Feature Requests-1190689 ] logging module '.' behavior Message-ID: Feature Requests item #1190689, was opened at 2005-04-26 20:19 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Vinay Sajip (vsajip) Summary: logging module '.' behavior Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 02:21 Message: Logged In: YES user_id=1267419 I am attaching a first pass it it. I've stored the "absolute" names everywhere, including both leading and trailing '.' I call this "absolute" by analogy with os.path.abspath(). I believe that similarities within the Python library help the user remember key concepts. What's missing: * I don't have aliases. * The "current working logger" is always '.' * And I haven't added any extra tags for the Formatter. But those are all very simple changes. The tough part is getting the path-searching correct. I have a big UnitTest suite which I can send to you if you'd like. The most important thing is that the word "root" is completely gone, but perhaps %(name)s should translate '.' to 'root' for backwards compatibility. The second-most important thing is that getLogger('.') returns the root logger. Third is that getLogger("Package.Module") is equivalent to getLogger(".Package.Module.") As for tags in the Formatter, after some testing I suggest these: %(name)s => abspath.rstrip('.'), but "." becomes "root" %(absname)s => abspath, with leading AND trailing dot, like a directory, so there is no question about whether the root displays as "." or "". It is always just dot in absolute notation. %(logger)s => abspath.rstrip('.'), maybe the prettiest I must tell you that, once I figured out how the logging module works, I really love it! Other possible additions: * Some useful, predefined filter classes: Never, OneTime (which must have a reset() method to clear its cache). I can send my version if you want. * A PipedStreamHandler. I'm not sure how to make this work. The idea is that, in UnitTesting, I want to read from some stream immediately after an operation, and I want the logged data to be immediately available, but to disappear as soon as I've read it. Does that make sense? Right now, I use a cStringIO object, with s.seek(0); s.truncate() after every read. -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-27 15:37 Message: Logged In: YES user_id=1267419 Novices always ask, "Why did it print 'root'? Where did that come from? After discussing this with some other "logging" module users, I think we've come up with a very good idea, which would maintain BACKWARDS COMPATIBILITY. Essentially, treat the logging module as a shell and the logger name as a path. Specifically, * Let the global logging functions operate on the "current worrking logger", which by default is "." (Let "root" be an alias for ".") * Change getLogger() so that it works on both absolute and relative logger paths. (Since the default current logger is "root", we maintain backwards compatibility.) * Change the format function so that %(name)s shows the relative path, if the absolute path starts with the current working logger name. * Add a format keyword, %(absname)s, which prints the absolute logger path. * Add another format keyword, %(logger)s, which prints what most people expect to see: the absolute logger name, sans the leading dot. (The "root" or "." logger would display as "", exactly the way it is usually accessed.) * Add global functions, change_current_logger() and get_current_logger(). * Add global function, alias(). Always create an alias for "root" to "." Examples:: from logging import * log = getLogger() #or getLogger(".") or getLogger("root") h1 = StreamHandler() f1 = Formatter("[%(name)s]%(message)s") h1.setFormatter(f1) log.addHandler(h1) h2 = StreamHandler() f2 = Formatter("[%(absname)s]%(message)s") h2.setFormatter(f2) log.addHandler(h2) h3 = StreamHandler() f3 = Formatter("[%(logger)s]%(message)s") h3.setFormatter(f3) log.addHandler(h3) log.error("First message") # ... child = getLogger("child") # or getLogger(".child") child.error("Bad news") This should print: [root]First message [.]First message []First message [child]Bad news [.child]Bad news [child]Bad news This would create tremendous flexibility, add some clarity to the meaning of the "root" logger, and still maintain complete backwards compatibility. I am willing to make the changes myself, including UnitTests, if there is agreement that they would be adopted. (Note that String.before() and String.after() would make the coding a little easier/clearer, but that's a different feature request.) -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Thu Apr 28 09:29:06 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 09:29:11 2005 Subject: [ python-Feature Requests-1190689 ] logging module '.' behavior Message-ID: Feature Requests item #1190689, was opened at 2005-04-26 20:19 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Vinay Sajip (vsajip) Summary: logging module '.' behavior Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 02:29 Message: Logged In: YES user_id=1267419 Oops. Where I wrote abspath.rstrip('.'), I meant abspath.strip('.') Drop both leading and trailing dots for the prettified path. -cdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 02:21 Message: Logged In: YES user_id=1267419 I am attaching a first pass it it. I've stored the "absolute" names everywhere, including both leading and trailing '.' I call this "absolute" by analogy with os.path.abspath(). I believe that similarities within the Python library help the user remember key concepts. What's missing: * I don't have aliases. * The "current working logger" is always '.' * And I haven't added any extra tags for the Formatter. But those are all very simple changes. The tough part is getting the path-searching correct. I have a big UnitTest suite which I can send to you if you'd like. The most important thing is that the word "root" is completely gone, but perhaps %(name)s should translate '.' to 'root' for backwards compatibility. The second-most important thing is that getLogger('.') returns the root logger. Third is that getLogger("Package.Module") is equivalent to getLogger(".Package.Module.") As for tags in the Formatter, after some testing I suggest these: %(name)s => abspath.rstrip('.'), but "." becomes "root" %(absname)s => abspath, with leading AND trailing dot, like a directory, so there is no question about whether the root displays as "." or "". It is always just dot in absolute notation. %(logger)s => abspath.rstrip('.'), maybe the prettiest I must tell you that, once I figured out how the logging module works, I really love it! Other possible additions: * Some useful, predefined filter classes: Never, OneTime (which must have a reset() method to clear its cache). I can send my version if you want. * A PipedStreamHandler. I'm not sure how to make this work. The idea is that, in UnitTesting, I want to read from some stream immediately after an operation, and I want the logged data to be immediately available, but to disappear as soon as I've read it. Does that make sense? Right now, I use a cStringIO object, with s.seek(0); s.truncate() after every read. -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-27 15:37 Message: Logged In: YES user_id=1267419 Novices always ask, "Why did it print 'root'? Where did that come from? After discussing this with some other "logging" module users, I think we've come up with a very good idea, which would maintain BACKWARDS COMPATIBILITY. Essentially, treat the logging module as a shell and the logger name as a path. Specifically, * Let the global logging functions operate on the "current worrking logger", which by default is "." (Let "root" be an alias for ".") * Change getLogger() so that it works on both absolute and relative logger paths. (Since the default current logger is "root", we maintain backwards compatibility.) * Change the format function so that %(name)s shows the relative path, if the absolute path starts with the current working logger name. * Add a format keyword, %(absname)s, which prints the absolute logger path. * Add another format keyword, %(logger)s, which prints what most people expect to see: the absolute logger name, sans the leading dot. (The "root" or "." logger would display as "", exactly the way it is usually accessed.) * Add global functions, change_current_logger() and get_current_logger(). * Add global function, alias(). Always create an alias for "root" to "." Examples:: from logging import * log = getLogger() #or getLogger(".") or getLogger("root") h1 = StreamHandler() f1 = Formatter("[%(name)s]%(message)s") h1.setFormatter(f1) log.addHandler(h1) h2 = StreamHandler() f2 = Formatter("[%(absname)s]%(message)s") h2.setFormatter(f2) log.addHandler(h2) h3 = StreamHandler() f3 = Formatter("[%(logger)s]%(message)s") h3.setFormatter(f3) log.addHandler(h3) log.error("First message") # ... child = getLogger("child") # or getLogger(".child") child.error("Bad news") This should print: [root]First message [.]First message []First message [child]Bad news [.child]Bad news [child]Bad news This would create tremendous flexibility, add some clarity to the meaning of the "root" logger, and still maintain complete backwards compatibility. I am willing to make the changes myself, including UnitTests, if there is agreement that they would be adopted. (Note that String.before() and String.after() would make the coding a little easier/clearer, but that's a different feature request.) -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Thu Apr 28 09:36:02 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 09:36:04 2005 Subject: [ python-Feature Requests-1190689 ] logging module root logger name Message-ID: Feature Requests item #1190689, was opened at 2005-04-27 01:19 Message generated for change (Comment added) made by vsajip You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Vinay Sajip (vsajip) >Summary: logging module root logger name Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- >Comment By: Vinay Sajip (vsajip) Date: 2005-04-28 07:36 Message: Logged In: YES user_id=308438 Logger names are hierarchical with dots separating levels in the hierarchy. So to me it does not make sense to have logger names which end in a dot, and you have given no reason why trailing dots should be supported. However, the hierarchy is not completely anologous to file system hierarchies - there is by design no concept of a "default" or "current" logger. I do not propose to make a change to this. However, I agree that the name of the root logger being "root" might be seen as unintuitive by some. Of your alternatives I think "logging" is best. I propose to add to the documentation the suggestion that users can define their own name for the root logger as in the following example: logging.getLogger().name = "myapp" People who use the root logger directly typically don't use other (named) loggers, because the whole point of using named loggers is to pinpoint areas of the application. Those users who use the root logger directly are typically not interested in finer granularity than the application or script itself. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 07:29 Message: Logged In: YES user_id=1267419 Oops. Where I wrote abspath.rstrip('.'), I meant abspath.strip('.') Drop both leading and trailing dots for the prettified path. -cdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 07:21 Message: Logged In: YES user_id=1267419 I am attaching a first pass it it. I've stored the "absolute" names everywhere, including both leading and trailing '.' I call this "absolute" by analogy with os.path.abspath(). I believe that similarities within the Python library help the user remember key concepts. What's missing: * I don't have aliases. * The "current working logger" is always '.' * And I haven't added any extra tags for the Formatter. But those are all very simple changes. The tough part is getting the path-searching correct. I have a big UnitTest suite which I can send to you if you'd like. The most important thing is that the word "root" is completely gone, but perhaps %(name)s should translate '.' to 'root' for backwards compatibility. The second-most important thing is that getLogger('.') returns the root logger. Third is that getLogger("Package.Module") is equivalent to getLogger(".Package.Module.") As for tags in the Formatter, after some testing I suggest these: %(name)s => abspath.rstrip('.'), but "." becomes "root" %(absname)s => abspath, with leading AND trailing dot, like a directory, so there is no question about whether the root displays as "." or "". It is always just dot in absolute notation. %(logger)s => abspath.rstrip('.'), maybe the prettiest I must tell you that, once I figured out how the logging module works, I really love it! Other possible additions: * Some useful, predefined filter classes: Never, OneTime (which must have a reset() method to clear its cache). I can send my version if you want. * A PipedStreamHandler. I'm not sure how to make this work. The idea is that, in UnitTesting, I want to read from some stream immediately after an operation, and I want the logged data to be immediately available, but to disappear as soon as I've read it. Does that make sense? Right now, I use a cStringIO object, with s.seek(0); s.truncate() after every read. -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-27 20:37 Message: Logged In: YES user_id=1267419 Novices always ask, "Why did it print 'root'? Where did that come from? After discussing this with some other "logging" module users, I think we've come up with a very good idea, which would maintain BACKWARDS COMPATIBILITY. Essentially, treat the logging module as a shell and the logger name as a path. Specifically, * Let the global logging functions operate on the "current worrking logger", which by default is "." (Let "root" be an alias for ".") * Change getLogger() so that it works on both absolute and relative logger paths. (Since the default current logger is "root", we maintain backwards compatibility.) * Change the format function so that %(name)s shows the relative path, if the absolute path starts with the current working logger name. * Add a format keyword, %(absname)s, which prints the absolute logger path. * Add another format keyword, %(logger)s, which prints what most people expect to see: the absolute logger name, sans the leading dot. (The "root" or "." logger would display as "", exactly the way it is usually accessed.) * Add global functions, change_current_logger() and get_current_logger(). * Add global function, alias(). Always create an alias for "root" to "." Examples:: from logging import * log = getLogger() #or getLogger(".") or getLogger("root") h1 = StreamHandler() f1 = Formatter("[%(name)s]%(message)s") h1.setFormatter(f1) log.addHandler(h1) h2 = StreamHandler() f2 = Formatter("[%(absname)s]%(message)s") h2.setFormatter(f2) log.addHandler(h2) h3 = StreamHandler() f3 = Formatter("[%(logger)s]%(message)s") h3.setFormatter(f3) log.addHandler(h3) log.error("First message") # ... child = getLogger("child") # or getLogger(".child") child.error("Bad news") This should print: [root]First message [.]First message []First message [child]Bad news [.child]Bad news [child]Bad news This would create tremendous flexibility, add some clarity to the meaning of the "root" logger, and still maintain complete backwards compatibility. I am willing to make the changes myself, including UnitTests, if there is agreement that they would be adopted. (Note that String.before() and String.after() would make the coding a little easier/clearer, but that's a different feature request.) -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Thu Apr 28 09:37:18 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 09:37:26 2005 Subject: [ python-Feature Requests-668905 ] logging module documentation Message-ID: Feature Requests item #668905, was opened at 2003-01-16 05:07 Message generated for change (Settings changed) made by vsajip You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=668905&group_id=5470 Category: Documentation Group: None >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Richard Jones (richard) Assigned to: Vinay Sajip (vsajip) Summary: logging module documentation Initial Comment: I believe the logging module documentation needs to give an example of how to simply log to a file. The following example snippet could be appropriate: import logging logger = logging.getLogger('myapp') hdlr = FileHandler('/var/myapp/debug.log') hdlr.setFormatter(Formatter('%(asctime)s %(level)s %(message)s')) logger.addHandler(hdlr) logger.setLevel(DEBUG) ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2004-03-16 04:04 Message: Logged In: YES user_id=6405 Accepting any file-like object, as the Twisted logging API does, would be nice. I wouldn't object to having to write "basicConfig(file('path/to/file'))" Thanks for looking into this Vinay! ---------------------------------------------------------------------- Comment By: Vinay Sajip (vsajip) Date: 2004-03-15 14:47 Message: Logged In: YES user_id=308438 I assume no one would pass an empty for filename or fmtstr, as this would make no sense. So I test for both None and the empty string at the same time using the "if value:" idiom. I have tried to be consistent with the view that wherever possible, logging should not generate spurious output (e.g. throwing exceptions), even when errors occur. ---------------------------------------------------------------------- Comment By: Drew Perttula (drewp) Date: 2004-03-15 13:19 Message: Logged In: YES user_id=127598 ITYM "if filename is not None" , "if fmtstr is None", and "Formatter(fmtstr)". I like the feature a lot, otherwise. For the record, the next most common things I do to a logger is to adjust its root's level to DEBUG but set the level of the channels I -don't- want to see back to INFO. (Therefore, any new channels will get output as DEBUG until I put them in the exclusion list.) I doubt this belongs in basicConfig, but if there became a convenience method to do it, I'd sure use it :) ---------------------------------------------------------------------- Comment By: Vinay Sajip (vsajip) Date: 2004-03-15 11:48 Message: Logged In: YES user_id=308438 How about if I modify basicConfig as follows? def basicConfig(filename=None, fmtstr=None, mode="w"): """ Do basic configuration for the logging system. If a filename is specified, create a FileHandler using it, otherwise create a StreamHandler. The created handler is added to the root logger. If a format string is specified, it will be used to initialize the Formatter - otherwise, a suitable default will be used. The mode is only used when a filename is specified, to determine how the file is to be opened. """ if len(root.handlers) == 0: if filename: hdlr = FileHandler(filename, mode) else: hdlr = StreamHandler() if not fmtstr: fmtstr = BASIC_FORMAT fmt = Formatter(fs) hdlr.setFormatter(fmt) root.addHandler(hdlr) ---------------------------------------------------------------------- Comment By: Richard Jones (richard) Date: 2004-03-09 21:21 Message: Logged In: YES user_id=6405 Please consider adding a simpler interface for people who just want to log to a file: import logging logger = logging.open('/var/log/myapp.log') which does the equivalent of: import logging logger = logging.getLogger('') hdlr = logging.FileHandler('/var/log/myapp.log') formatter = logging.Formatter('%(asctime)s %(levelname)s %(message)s') hdlr.setFormatter(formatter) logger.addHandler(hdlr) logger.setLevel(logging.INFO) which is what I expect most users will want when they first see this module. If they then want to mess around with additional handers, or different formatting or whatever, then they can read the reference doc. Please make the simplest use-case much simpler! ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-01-01 07:10 Message: Logged In: YES user_id=80475 I would also like to see a more thorough set of introductory examples. The second page of the unittest documentation could serve as a model -- it attempts to cover the 90% of what you need to know in a single page of examples. As it stands now, the documentation for the logging module is formidable. Those wanting to take advantage of the module face a steep learning curve from rather heavyweight documentation -- it's complexity exceeds that of almost all modules except for the email and xml packages. Assigning to Vinay Sajip to work with Anthony to improve the docs. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2003-07-08 06:32 Message: Logged In: YES user_id=29957 Why was this closed? nnorwitz's doc fixes, as seen in CVS or http://www.python.org/dev/doc/devel/lib/module-logging.html have no examples section. Running the current logging module docs past a number of python coders here produced a consistent "what the heck?" response - the opening introduction provides no indications of how to use the package. This is a problem for me, right now, so I'm re-opening and assigning to myself to fix. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-28 06:37 Message: Logged In: YES user_id=80475 Reviewed. Closing RFE. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-01-25 21:36 Message: Logged In: YES user_id=33168 I just updated the logging documentation. Could you please review for accuracy and completeness? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=668905&group_id=5470 From noreply at sourceforge.net Thu Apr 28 09:43:24 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 09:43:26 2005 Subject: [ python-Feature Requests-1190689 ] logging module root logger name Message-ID: Feature Requests item #1190689, was opened at 2005-04-27 01:19 Message generated for change (Comment added) made by vsajip You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Vinay Sajip (vsajip) Summary: logging module root logger name Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- >Comment By: Vinay Sajip (vsajip) Date: 2005-04-28 07:43 Message: Logged In: YES user_id=308438 Whoops! I don't quite know what happened, but I think both of us were updating this RFE entry concurrently. I only saw your followup starting "Novices always ask..." before I posted my response. ---------------------------------------------------------------------- Comment By: Vinay Sajip (vsajip) Date: 2005-04-28 07:36 Message: Logged In: YES user_id=308438 Logger names are hierarchical with dots separating levels in the hierarchy. So to me it does not make sense to have logger names which end in a dot, and you have given no reason why trailing dots should be supported. However, the hierarchy is not completely anologous to file system hierarchies - there is by design no concept of a "default" or "current" logger. I do not propose to make a change to this. However, I agree that the name of the root logger being "root" might be seen as unintuitive by some. Of your alternatives I think "logging" is best. I propose to add to the documentation the suggestion that users can define their own name for the root logger as in the following example: logging.getLogger().name = "myapp" People who use the root logger directly typically don't use other (named) loggers, because the whole point of using named loggers is to pinpoint areas of the application. Those users who use the root logger directly are typically not interested in finer granularity than the application or script itself. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 07:29 Message: Logged In: YES user_id=1267419 Oops. Where I wrote abspath.rstrip('.'), I meant abspath.strip('.') Drop both leading and trailing dots for the prettified path. -cdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 07:21 Message: Logged In: YES user_id=1267419 I am attaching a first pass it it. I've stored the "absolute" names everywhere, including both leading and trailing '.' I call this "absolute" by analogy with os.path.abspath(). I believe that similarities within the Python library help the user remember key concepts. What's missing: * I don't have aliases. * The "current working logger" is always '.' * And I haven't added any extra tags for the Formatter. But those are all very simple changes. The tough part is getting the path-searching correct. I have a big UnitTest suite which I can send to you if you'd like. The most important thing is that the word "root" is completely gone, but perhaps %(name)s should translate '.' to 'root' for backwards compatibility. The second-most important thing is that getLogger('.') returns the root logger. Third is that getLogger("Package.Module") is equivalent to getLogger(".Package.Module.") As for tags in the Formatter, after some testing I suggest these: %(name)s => abspath.rstrip('.'), but "." becomes "root" %(absname)s => abspath, with leading AND trailing dot, like a directory, so there is no question about whether the root displays as "." or "". It is always just dot in absolute notation. %(logger)s => abspath.rstrip('.'), maybe the prettiest I must tell you that, once I figured out how the logging module works, I really love it! Other possible additions: * Some useful, predefined filter classes: Never, OneTime (which must have a reset() method to clear its cache). I can send my version if you want. * A PipedStreamHandler. I'm not sure how to make this work. The idea is that, in UnitTesting, I want to read from some stream immediately after an operation, and I want the logged data to be immediately available, but to disappear as soon as I've read it. Does that make sense? Right now, I use a cStringIO object, with s.seek(0); s.truncate() after every read. -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-27 20:37 Message: Logged In: YES user_id=1267419 Novices always ask, "Why did it print 'root'? Where did that come from? After discussing this with some other "logging" module users, I think we've come up with a very good idea, which would maintain BACKWARDS COMPATIBILITY. Essentially, treat the logging module as a shell and the logger name as a path. Specifically, * Let the global logging functions operate on the "current worrking logger", which by default is "." (Let "root" be an alias for ".") * Change getLogger() so that it works on both absolute and relative logger paths. (Since the default current logger is "root", we maintain backwards compatibility.) * Change the format function so that %(name)s shows the relative path, if the absolute path starts with the current working logger name. * Add a format keyword, %(absname)s, which prints the absolute logger path. * Add another format keyword, %(logger)s, which prints what most people expect to see: the absolute logger name, sans the leading dot. (The "root" or "." logger would display as "", exactly the way it is usually accessed.) * Add global functions, change_current_logger() and get_current_logger(). * Add global function, alias(). Always create an alias for "root" to "." Examples:: from logging import * log = getLogger() #or getLogger(".") or getLogger("root") h1 = StreamHandler() f1 = Formatter("[%(name)s]%(message)s") h1.setFormatter(f1) log.addHandler(h1) h2 = StreamHandler() f2 = Formatter("[%(absname)s]%(message)s") h2.setFormatter(f2) log.addHandler(h2) h3 = StreamHandler() f3 = Formatter("[%(logger)s]%(message)s") h3.setFormatter(f3) log.addHandler(h3) log.error("First message") # ... child = getLogger("child") # or getLogger(".child") child.error("Bad news") This should print: [root]First message [.]First message []First message [child]Bad news [.child]Bad news [child]Bad news This would create tremendous flexibility, add some clarity to the meaning of the "root" logger, and still maintain complete backwards compatibility. I am willing to make the changes myself, including UnitTests, if there is agreement that they would be adopted. (Note that String.before() and String.after() would make the coding a little easier/clearer, but that's a different feature request.) -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Thu Apr 28 10:31:51 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Apr 28 10:31:53 2005 Subject: [ python-Feature Requests-1190701 ] Add 'before' and 'after' methods to Strings Message-ID: Feature Requests item #1190701, was opened at 2005-04-26 20:35 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Rejected >Priority: 1 Submitted By: Christopher Dunn (cxdunn) Assigned to: Nobody/Anonymous (nobody) Summary: Add 'before' and 'after' methods to Strings Initial Comment: GNU String used to have two very useful methods, 'before' and 'after'. These are so useful I keep them defined in an __init__.py file. (Unfortunately, I do not know how to make them methods, instead of global functions.) Usage: >>> "root.sub".before(".") 'root' >>> "root.sub1.sub2".after("root.sub1") '.sub2' They work like s.split(word)[0], and s.split(word)[-1], but they are so intuitive they ought to be part of the interface. I'm not sure whether they should raise exceptions on failure, or simply return the whole string. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 03:31 Message: Logged In: YES user_id=1267419 I guess you're right. My goal here is to move my company from Tcl to Python, and there are surely more important inducements than an expanded string class. I will try to find out what "I'm -1" means. For what it's worth, here is a reference to the old libg++ GNU String library: http://www.math.utah.edu/docs/info/libg++_19.html z = x.before("o") sets z to the part of x to the left of the first occurrence of "o", or "Hell" in this case. The argument may also be a String, SubString, or Regex. (If there is no match, z is set to "".) x.before("ll") = "Bri"; sets the part of x to the left of "ll" to "Bri", setting x to "Brillo". z = x.before(2) sets z to the part of x to the left of x[2], or "He" in this case. z = x.after("Hel") sets z to the part of x to the right of "Hel", or "lo" in this case. z = x.through("el") sets z to the part of x up and including "el", or "Hel" in this case. z = x.from("el") sets z to the part of x from "el" to the end, or "ello" in this case. x.after("Hel") = "p"; sets x to "Help"; z = x.after(3) sets z to the part of x to the right of x[3] or "o" in this case. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 01:58 Message: Logged In: YES user_id=80475 You read too much into a simplified example. Test the find() result for -1 and be done with it. Go ahead with a cookbook recipe if you think that no one else is bright enough to write their own. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:54 Message: Logged In: YES user_id=1267419 Sorry for the double-post. I thought I'd lost it and re-typed the whole thing. Read the top one only -- less buggy. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:50 Message: Logged In: YES user_id=1267419 Your examples prove my point:: >>> s = "Monty.Python" >>> t = "fubar" >>> s[:s.find(t)] 'Monty.Pytho' >>> s[s.find(t)+len(t):] 'y.Python' Of course, this would work: >>> s.split(t)[0] 'Monty.Python' >>> s.split(t)[-1] 'Monty.Python' That is not terrible, but the behavior I want is actually more like strip()/rstrip():: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] Besides, it's a question of elegance. These are very useful little functions, which look wonderful as methods of string, and the on-the-fly solutions are prone to error. Reconsider? If not, I'll just post it to the Cookbook (without your name -- I'm not trying to embarrass anyone) to point out the danger of relying on string.find(). -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 01:40 Message: Logged In: YES user_id=1267419 Your examples prove my point: >>> s = "root.sub" >>> t = "fubar" >>> s[:s.find(t)] 'root.su' >>> s = "root.sub1.sub2" >>> t = "fubar" >>> s[s.find(sep)+len(sep):] '.sub1.sub2' string.find() is the wrong way. I can live with string.split(): >>> "root.sub1.sub2" >>> t = '.' >>> s.split(t)[0] 'root' >>> s.split(t)[-1] 'sub2' >>> t = "fubar" >>> s.split(t)[0] 'root.sub1.sub2' >>> s.split(t)[-1] 'root.sub1.sub2' This is not terrible, but the desired behavior is really more like strip/rstrip:: def before( s, first ): """Find first inside string s and return everything before that. >>> before('xyz.pdq.abc', '.') 'xyz' >>> before('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(first)[0] def after( s, first ): """Find first inside string s and return everything after that. >>> after('xyz.pdq.abc', '.') 'pdq.abc' >>> after('xyz.pdq', 'xyz.') 'pdq' >>> after('xyz.pdq.abc', 'fubar') '' """ return first.join(s.split(first)[1:]) def rbefore( s, last ): """Find last inside string s, from the right, and return everything before that. >>> rbefore('xyz.pdq.abc', '.') 'xyz.pdq' >>> rbefore('xyz.pdq.abc', 'fubar') '' """ return last.join(s.split(last)[:-1]) def rafter( s, last ): """Find last inside string s, from the right and return everything after that. >>> rafter('xyz.pdq.abc', '.') 'abc' >>> rafter('xyz.pdq.abc', 'fubar') 'xyz.pdq.abc' """ return s.split(last)[-1] It's a question of elegance. These are very useful, infuitive functions, and I cannot add them to string myself. And as you've seen, it's easy to create bugs when you try to do this on the fly. Reconsider? If not, I'll just post it in the Cookbook, to point out the dangers of relying on string.find. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 00:15 Message: Logged In: YES user_id=80475 I'm -1 on expanding the string API for something so easily coded with existing primitives: >>> s = "root.sub" >>> t = "." >>> s[:s.find(t)] 'root' >>> s = "root.sub1.sub2" >>> t = "root.sub1" >>> s[s.find(sep)+len(sep):] 'sub1.sub2' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190701&group_id=5470 From noreply at sourceforge.net Thu Apr 28 14:00:21 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:10:21 2005 Subject: [ python-Bugs-1191043 ] bz2 RuntimeError when decompressing file Message-ID: Bugs item #1191043, was opened at 2005-04-27 10:34 Message generated for change (Comment added) made by catlee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris AtLee (catlee) Assigned to: Nobody/Anonymous (nobody) Summary: bz2 RuntimeError when decompressing file Initial Comment: The following code: echo -n Testing123 | bzip2 > test.bz2 python -c "import bz2; lines = bz2.BZ2File('test.bz2').readlines()" produces this output: Traceback (most recent call last): File "", line 1, in ? RuntimeError: wrong sequence of bz2 library commands used Tested on Python 2.4.1 (debian unstable - April 1 2005), and Python 2.3.5 (debian unstable - May 26 2005) ---------------------------------------------------------------------- >Comment By: Chris AtLee (catlee) Date: 2005-04-28 08:00 Message: Logged In: YES user_id=186532 How is test.bz2 not a valid bz2 file? The command line tool "bzcat" or "bunzip2" can operate properly on it. Using BZ2File("test.bz2").read() works properly, it's just the readlines() call that breaks. Try this out: import bz2 bz2.BZ2File("test.bz2","w").write("testing123") # This works fine assert bz2.BZ2File("test.bz2").read() == "testing123" # This raises a RuntimeError assert bz2.BZ2File("test.bz2").readlines() == ["testing123"] ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 03:06 Message: Logged In: YES user_id=80475 The looks like correct behavior to me. The test.bz2 file is not in a valid bz2 format. I suspect that you've misread the BZ2File API which is intended for reading and writing uncompressed data to and from a file in a bz2 format (where the data is stored in compressed form). If this interpretation of the bug report is correct, please mark as not-a-bug and close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 From noreply at sourceforge.net Thu Apr 28 21:23:33 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:10:24 2005 Subject: [ python-Feature Requests-1190689 ] logging module root logger name Message-ID: Feature Requests item #1190689, was opened at 2005-04-26 20:19 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 Category: Python Library Group: None Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: Christopher Dunn (cxdunn) Assigned to: Vinay Sajip (vsajip) Summary: logging module root logger name Initial Comment: I would like a trailing '.' to be ignored in names passed to getLogger(), like a trainling '/' in a Unix path. In module 'foo': logfoo = getLogger('.foo.') # logger '"" should be the parent of ".foo" Elsewhere, controlled by the user of that module: import foo logdefault = getLogger('.') hdlr = StreamHandler() fmtr = Formatter("%(name)s:%(msg)s") hdlr.setFormatter(fmtr) logdefault.addHandler(hdlr) Given this change, I would also like the name of the default logger to be displayed as '.', or even "", rather than 'root'. The current behavior is odd: logfoo.info("Foo message") displays .foo:Foo message buf logdefault.info("Default message") displays root:Default message I NEVER mentioned the word "root" anywhere! And I don't think it's very descriptive. I would rather see ANY of these: :Default message .:Default message default:Default message logging:Default message These changes would make the system more intuitive. -cxdunn ---------------------------------------------------------------------- >Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 14:23 Message: Logged In: YES user_id=1267419 You're right! That works like a charm: >>> import logging >>> logging.getLogger().name = '.' >>> logging.warning("I am root") WARNING:.:I am root >>> sub = logging.getLogger('.sub') >>> sub.warning("I am a child") WARNING:.sub:I am a child Setting the root to "" also works: >>> import logging >>> logging.getLogger().name = "" >>> logging.warning("I am root") WARNING::I am root >>> sub = logging.getLogger('sub') >>> sub.warning("I am a child") WARNING:sub:I am a child I agree with your other points. The flexibility would add little value. I brought this issue up b/c I was confused by the docs. Clearly, But it is not so clear that "A" is a child of "root". * "A.B" is obviously a child of "A" * ".A" is *clearly* a child of "." * And "A" is presumably a child of "". * But it is not so clear that "A" is a child of "root" Since *everything* is a child of the root logger, that's worth reiterating in the docs. And if there is truly only 1 root logger, then it should be possible to find it by name: >>> import logging >>> logging.getLogger().name ="." >>> logging.warning("I am root") WARNING:.:I am root >>> unknown = logging.getLogger(".") >>> unknown.warning("Who am I?") WARNING:.:Who am I? >>> unknown == logging.getLogger() False In fact: >>> import logging >>> logging.getLogger() == logging.getLogger() #just a test True >>> logging.getLogger() == logging.getLogger("root") #should be same! False This is not an easy module to understand, but it's amazingly powerful. One last suggestion. You have logging.handlers. You could also have logging.filters. For example: class Opaque(Filter): """A simple way to prevent any messages from getting through.""" def __init__(self, name=None): pass def filter(self, rec): return False class Unique(Filter): """Messages are allowed through just once. The 'message' includes substitutions, but is not formatted by the handler. If it were, then practically all messages would be unique! """ def __init__(self, name=""): Filter.__init__(self, name) self.reset() def reset(self): """Act as if nothing has happened.""" self.__logged = {} def filter(self, rec): return Filter.filter(self, rec) and self.__is_first_time(rec) def __is_first_time(self, rec): """Emit a message only once.""" msg = rec.msg %(rec.args) if msg in self.__logged: self.__logged[msg] += 1 return False else: self.__logged[msg] = 1 return True Actually, this might be Cookbook material. I'll write it up. Thanks for your time. -cxdunn ---------------------------------------------------------------------- Comment By: Vinay Sajip (vsajip) Date: 2005-04-28 02:43 Message: Logged In: YES user_id=308438 Whoops! I don't quite know what happened, but I think both of us were updating this RFE entry concurrently. I only saw your followup starting "Novices always ask..." before I posted my response. ---------------------------------------------------------------------- Comment By: Vinay Sajip (vsajip) Date: 2005-04-28 02:36 Message: Logged In: YES user_id=308438 Logger names are hierarchical with dots separating levels in the hierarchy. So to me it does not make sense to have logger names which end in a dot, and you have given no reason why trailing dots should be supported. However, the hierarchy is not completely anologous to file system hierarchies - there is by design no concept of a "default" or "current" logger. I do not propose to make a change to this. However, I agree that the name of the root logger being "root" might be seen as unintuitive by some. Of your alternatives I think "logging" is best. I propose to add to the documentation the suggestion that users can define their own name for the root logger as in the following example: logging.getLogger().name = "myapp" People who use the root logger directly typically don't use other (named) loggers, because the whole point of using named loggers is to pinpoint areas of the application. Those users who use the root logger directly are typically not interested in finer granularity than the application or script itself. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 02:29 Message: Logged In: YES user_id=1267419 Oops. Where I wrote abspath.rstrip('.'), I meant abspath.strip('.') Drop both leading and trailing dots for the prettified path. -cdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 02:21 Message: Logged In: YES user_id=1267419 I am attaching a first pass it it. I've stored the "absolute" names everywhere, including both leading and trailing '.' I call this "absolute" by analogy with os.path.abspath(). I believe that similarities within the Python library help the user remember key concepts. What's missing: * I don't have aliases. * The "current working logger" is always '.' * And I haven't added any extra tags for the Formatter. But those are all very simple changes. The tough part is getting the path-searching correct. I have a big UnitTest suite which I can send to you if you'd like. The most important thing is that the word "root" is completely gone, but perhaps %(name)s should translate '.' to 'root' for backwards compatibility. The second-most important thing is that getLogger('.') returns the root logger. Third is that getLogger("Package.Module") is equivalent to getLogger(".Package.Module.") As for tags in the Formatter, after some testing I suggest these: %(name)s => abspath.rstrip('.'), but "." becomes "root" %(absname)s => abspath, with leading AND trailing dot, like a directory, so there is no question about whether the root displays as "." or "". It is always just dot in absolute notation. %(logger)s => abspath.rstrip('.'), maybe the prettiest I must tell you that, once I figured out how the logging module works, I really love it! Other possible additions: * Some useful, predefined filter classes: Never, OneTime (which must have a reset() method to clear its cache). I can send my version if you want. * A PipedStreamHandler. I'm not sure how to make this work. The idea is that, in UnitTesting, I want to read from some stream immediately after an operation, and I want the logged data to be immediately available, but to disappear as soon as I've read it. Does that make sense? Right now, I use a cStringIO object, with s.seek(0); s.truncate() after every read. -cxdunn ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-27 15:37 Message: Logged In: YES user_id=1267419 Novices always ask, "Why did it print 'root'? Where did that come from? After discussing this with some other "logging" module users, I think we've come up with a very good idea, which would maintain BACKWARDS COMPATIBILITY. Essentially, treat the logging module as a shell and the logger name as a path. Specifically, * Let the global logging functions operate on the "current worrking logger", which by default is "." (Let "root" be an alias for ".") * Change getLogger() so that it works on both absolute and relative logger paths. (Since the default current logger is "root", we maintain backwards compatibility.) * Change the format function so that %(name)s shows the relative path, if the absolute path starts with the current working logger name. * Add a format keyword, %(absname)s, which prints the absolute logger path. * Add another format keyword, %(logger)s, which prints what most people expect to see: the absolute logger name, sans the leading dot. (The "root" or "." logger would display as "", exactly the way it is usually accessed.) * Add global functions, change_current_logger() and get_current_logger(). * Add global function, alias(). Always create an alias for "root" to "." Examples:: from logging import * log = getLogger() #or getLogger(".") or getLogger("root") h1 = StreamHandler() f1 = Formatter("[%(name)s]%(message)s") h1.setFormatter(f1) log.addHandler(h1) h2 = StreamHandler() f2 = Formatter("[%(absname)s]%(message)s") h2.setFormatter(f2) log.addHandler(h2) h3 = StreamHandler() f3 = Formatter("[%(logger)s]%(message)s") h3.setFormatter(f3) log.addHandler(h3) log.error("First message") # ... child = getLogger("child") # or getLogger(".child") child.error("Bad news") This should print: [root]First message [.]First message []First message [child]Bad news [.child]Bad news [child]Bad news This would create tremendous flexibility, add some clarity to the meaning of the "root" logger, and still maintain complete backwards compatibility. I am willing to make the changes myself, including UnitTests, if there is agreement that they would be adopted. (Note that String.before() and String.after() would make the coding a little easier/clearer, but that's a different feature request.) -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190689&group_id=5470 From noreply at sourceforge.net Thu Apr 28 14:36:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:41:14 2005 Subject: [ python-Bugs-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Bugs item #1190596, was opened at 2005-04-26 17:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:36 Message: Logged In: YES user_id=80475 Write a simple wrapper to save and restore the global state: >>> def mymonthcalendar(year, month, first=None): if first is None: return monthcalendar(year, month) saved = firstweekday() setfirstweekday(first) rows = monthcalendar(year, month) setfirstweekday(saved) return rows That is essentially what the existing API is for. For a multi-threaded environment, you would need to add locks to th e critical section in the above code (the same would be true for your proposed patch). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:24 Message: Logged In: YES user_id=127625 Different users of a long-running Python process may have different preferences for the first weekday; if the application tries to use the setfirstweekday() API for each of them, then they end up racing to set it. Making the option selectable per-call makes it side-effect free, and therefore more robust for multi-user applications. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 06:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 06:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 01:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 From noreply at sourceforge.net Thu Apr 28 15:44:34 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:41:16 2005 Subject: [ python-Feature Requests-1191699 ] make slices pickable Message-ID: Feature Requests item #1191699, was opened at 2005-04-28 13:44 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sebastien de Menten (sdementen) Assigned to: Nobody/Anonymous (nobody) Summary: make slices pickable Initial Comment: As suggested by the summary, provide pickability of slices by default. Currently one has to use copy_reg module to register slices as pickables. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470 From noreply at sourceforge.net Thu Apr 28 15:42:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:41:17 2005 Subject: [ python-Feature Requests-1191697 ] slice indices different than integers Message-ID: Feature Requests item #1191697, was opened at 2005-04-28 13:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191697&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sebastien de Menten (sdementen) Assigned to: Nobody/Anonymous (nobody) Summary: slice indices different than integers Initial Comment: Hi, Slice notation is quite convenient while addressing some item in a collection. It would be nice to extend this notation by enabling any object instead of integer in slices. Example 1: let time_serie be an object representing a time serie and date and days object that manages dates and concept of days interval, one could indexed time_serie with: time_serie[date(2004,3,4):date(2004,5,3)] or time_serie[date(2004,3,4):date(2004,5,3):days(5)] Example 2: Let f be a numerical function with multiple arguments, one could get an array of results by using f[3:10:2, 1:5:3] or naturally :-) f[3.2:10.1:0.4, 1:5:3] Well, I think it is a matter of removing the check on argument of slices to enable the syntactic sugar start:end:step in __getitem__ calls as well as adding integer checks on slice attributes when using it in old way. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191697&group_id=5470 From noreply at sourceforge.net Thu Apr 28 15:02:11 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:41:19 2005 Subject: [ python-Bugs-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Bugs item #1190596, was opened at 2005-04-26 18:10 Message generated for change (Comment added) made by tseaver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Tres Seaver (tseaver) Date: 2005-04-28 09:02 Message: Logged In: YES user_id=127625 WIth the patch, if you pass 'firstweekday' when calling 'monthcalendar' or 'weekheader', there is no race, and therefore no need to guard the global, because the global is ignored. Nothing else in the module depends on the value of that global except those two functions. I'm attaching a doctest which demonstrates the use case. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 08:36 Message: Logged In: YES user_id=80475 Write a simple wrapper to save and restore the global state: >>> def mymonthcalendar(year, month, first=None): if first is None: return monthcalendar(year, month) saved = firstweekday() setfirstweekday(first) rows = monthcalendar(year, month) setfirstweekday(saved) return rows That is essentially what the existing API is for. For a multi-threaded environment, you would need to add locks to th e critical section in the above code (the same would be true for your proposed patch). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 08:24 Message: Logged In: YES user_id=127625 Different users of a long-running Python process may have different preferences for the first weekday; if the application tries to use the setfirstweekday() API for each of them, then they end up racing to set it. Making the option selectable per-call makes it side-effect free, and therefore more robust for multi-user applications. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 From noreply at sourceforge.net Thu Apr 28 14:39:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:41:21 2005 Subject: [ python-Feature Requests-1190033 ] The array module and the buffer interface Message-ID: Feature Requests item #1190033, was opened at 2005-04-25 23:59 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Josiah Carlson (josiahcarlson) Assigned to: Nobody/Anonymous (nobody) Summary: The array module and the buffer interface Initial Comment: It would be terribly convenient if array objects were willing to take any object supporting the buffer interface as initialization or extension via a.fromstring(). They currently offer the buffer interface for other objects to read/write once the array has been created, but they do not accept buffers during creation or extension (except for the typecode 'c'). ---------------------------------------------------------------------- >Comment By: Josiah Carlson (josiahcarlson) Date: 2005-04-28 05:39 Message: Logged In: YES user_id=341410 I couldn't sleep, so I started working on it. Note: a.fromstring(buf_obj) works in Python 2.3 and 2.4, it just isn't documented, nor is it exposed via array_new. I've got less than 20 lines of changes that seem to be necessary for array(typecode, buf) to be supported, including documentation and unittest updates. I'll sleep on it (I'm getting tired) and upload it tomorrow. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-04-27 23:37 Message: Logged In: YES user_id=341410 Not right now, but I could probably have one for you tomorrow. You want doc updates and tests too? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-27 22:20 Message: Logged In: YES user_id=80475 Do you have a patch? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190033&group_id=5470 From noreply at sourceforge.net Thu Apr 28 22:40:52 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:50:08 2005 Subject: [ python-Feature Requests-1191964 ] asynchronous Subprocess Message-ID: Feature Requests item #1191964, was opened at 2005-04-28 13:40 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191964&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Josiah Carlson (josiahcarlson) Assigned to: Nobody/Anonymous (nobody) Summary: asynchronous Subprocess Initial Comment: It would be terribly nice if the Popen class in the subprocess module would allow a programmer to easily say "send some data right now (if I have some to send) and receive whatever information is available right now". Essentially the equivalent of asyncore.loop(count=1), only that it returns the data received, instead of placing it in a buffer. Why would this functionality be useful? Currently, when using the subprocess module with pipes, the interaction with a pipe is limited to "send data if desired, close the subprocess' stdin, read all output from the subprocess' stdout, ...". Certainly one can pull the pipes out of the Popen instance, and perform the necessary functions on posix systems (with select or poll), but the additional magic on WIndows is a little less obvious (but still possible). There is a post by Paul Du Bois with an example using win32pipe.PeekNamedPipe: http://groups-beta.google.com/group/comp.lang.python/msg/115e9332cc1ca09d?hl=en And a message from Neil Hodgeson stating that PeekNamedPipe works on anonymous pipes: http://mail.python.org/pipermail/python-dev/2000-May/004200.html With this modification, creating Expect-like modules for any platform, as well as embedded shells inside any GUI with a text input widget, becomes easy. Heck, even Idle could use it rather than a socket for its interactive interpreter. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191964&group_id=5470 From noreply at sourceforge.net Thu Apr 28 22:13:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:50:10 2005 Subject: [ python-Feature Requests-1185383 ] Make bisect.* functions accept an optional compare function Message-ID: Feature Requests item #1185383, was opened at 2005-04-18 21:26 Message generated for change (Comment added) made by jonaskoelker You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1185383&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Marcin Ciura (mciura) Assigned to: Nobody/Anonymous (nobody) Summary: Make bisect.* functions accept an optional compare function Initial Comment: The usability of bisect.bisect_right, bisect.bisect_left, bisect.insort_right and bisect.insort_left would increase if they accepted an optional less_than function to compare the keys. Here is my use case: I have a sorted list of reversed words and a parallel list of flags associated with these words (taken from ispell). The list of reversed words is the one searched; the flags are the result of a search. Issue #1: Now I cannot use simply a list of tuples (rev_word,flags) and a custom less_than function that compares only the first elements of two tuples. Issue #2: When a word is not found in the list, I'd like to make an educated guess as to its flags (this makes more sense in non-English languages, where e.g. infinitives have a special ending), using bisect_left and bisect_right: from bisect import * less_than = lambda x,y: x[:3] Bugs item #1190596, was opened at 2005-04-26 18:10 Message generated for change (Comment added) made by tseaver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Tres Seaver (tseaver) Date: 2005-04-28 08:24 Message: Logged In: YES user_id=127625 Different users of a long-running Python process may have different preferences for the first weekday; if the application tries to use the setfirstweekday() API for each of them, then they end up racing to set it. Making the option selectable per-call makes it side-effect free, and therefore more robust for multi-user applications. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 From noreply at sourceforge.net Thu Apr 28 14:14:28 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:50:15 2005 Subject: [ python-Bugs-1191043 ] bz2 RuntimeError when decompressing file Message-ID: Bugs item #1191043, was opened at 2005-04-27 09:34 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 Category: Python Library Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris AtLee (catlee) >Assigned to: Raymond Hettinger (rhettinger) Summary: bz2 RuntimeError when decompressing file Initial Comment: The following code: echo -n Testing123 | bzip2 > test.bz2 python -c "import bz2; lines = bz2.BZ2File('test.bz2').readlines()" produces this output: Traceback (most recent call last): File "", line 1, in ? RuntimeError: wrong sequence of bz2 library commands used Tested on Python 2.4.1 (debian unstable - April 1 2005), and Python 2.3.5 (debian unstable - May 26 2005) ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:14 Message: Logged In: YES user_id=80475 Okay, I see. Will look into it. ---------------------------------------------------------------------- Comment By: Chris AtLee (catlee) Date: 2005-04-28 07:00 Message: Logged In: YES user_id=186532 How is test.bz2 not a valid bz2 file? The command line tool "bzcat" or "bunzip2" can operate properly on it. Using BZ2File("test.bz2").read() works properly, it's just the readlines() call that breaks. Try this out: import bz2 bz2.BZ2File("test.bz2","w").write("testing123") # This works fine assert bz2.BZ2File("test.bz2").read() == "testing123" # This raises a RuntimeError assert bz2.BZ2File("test.bz2").readlines() == ["testing123"] ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:06 Message: Logged In: YES user_id=80475 The looks like correct behavior to me. The test.bz2 file is not in a valid bz2 format. I suspect that you've misread the BZ2File API which is intended for reading and writing uncompressed data to and from a file in a bz2 format (where the data is stored in compressed form). If this interpretation of the bug report is correct, please mark as not-a-bug and close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191043&group_id=5470 From noreply at sourceforge.net Thu Apr 28 13:42:36 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:50:18 2005 Subject: [ python-Bugs-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Bugs item #1190596, was opened at 2005-04-26 17:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 06:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 06:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 01:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 From noreply at sourceforge.net Thu Apr 28 13:06:15 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:50:19 2005 Subject: [ python-Bugs-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Bugs item #1190596, was opened at 2005-04-26 18:10 Message generated for change (Comment added) made by tseaver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1190596&group_id=5470 From noreply at sourceforge.net Fri Apr 29 00:51:23 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:50:26 2005 Subject: [ python-Feature Requests-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Feature Requests item #1190596, was opened at 2005-04-26 17:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 >Category: Python Library >Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) >Assigned to: Tim Peters (tim_one) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 17:51 Message: Logged In: YES user_id=80475 Tim, I'm -0 on this one. Do you have an opinion? Essentially, the OP wants an alternate, thread-safe interface to specifying firstweekday on a per-call basis. It can already be done with a wrapper function and a lock, so it is just a convenience issue. ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 08:02 Message: Logged In: YES user_id=127625 WIth the patch, if you pass 'firstweekday' when calling 'monthcalendar' or 'weekheader', there is no race, and therefore no need to guard the global, because the global is ignored. Nothing else in the module depends on the value of that global except those two functions. I'm attaching a doctest which demonstrates the use case. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:36 Message: Logged In: YES user_id=80475 Write a simple wrapper to save and restore the global state: >>> def mymonthcalendar(year, month, first=None): if first is None: return monthcalendar(year, month) saved = firstweekday() setfirstweekday(first) rows = monthcalendar(year, month) setfirstweekday(saved) return rows That is essentially what the existing API is for. For a multi-threaded environment, you would need to add locks to th e critical section in the above code (the same would be true for your proposed patch). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:24 Message: Logged In: YES user_id=127625 Different users of a long-running Python process may have different preferences for the first weekday; if the application tries to use the setfirstweekday() API for each of them, then they end up racing to set it. Making the option selectable per-call makes it side-effect free, and therefore more robust for multi-user applications. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 06:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 06:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 01:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 From noreply at sourceforge.net Fri Apr 29 01:38:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 02:51:34 2005 Subject: [ python-Feature Requests-1191964 ] asynchronous Subprocess Message-ID: Feature Requests item #1191964, was opened at 2005-04-28 13:40 Message generated for change (Comment added) made by parameter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191964&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Josiah Carlson (josiahcarlson) Assigned to: Nobody/Anonymous (nobody) Summary: asynchronous Subprocess Initial Comment: It would be terribly nice if the Popen class in the subprocess module would allow a programmer to easily say "send some data right now (if I have some to send) and receive whatever information is available right now". Essentially the equivalent of asyncore.loop(count=1), only that it returns the data received, instead of placing it in a buffer. Why would this functionality be useful? Currently, when using the subprocess module with pipes, the interaction with a pipe is limited to "send data if desired, close the subprocess' stdin, read all output from the subprocess' stdout, ...". Certainly one can pull the pipes out of the Popen instance, and perform the necessary functions on posix systems (with select or poll), but the additional magic on WIndows is a little less obvious (but still possible). There is a post by Paul Du Bois with an example using win32pipe.PeekNamedPipe: http://groups-beta.google.com/group/comp.lang.python/msg/115e9332cc1ca09d?hl=en And a message from Neil Hodgeson stating that PeekNamedPipe works on anonymous pipes: http://mail.python.org/pipermail/python-dev/2000-May/004200.html With this modification, creating Expect-like modules for any platform, as well as embedded shells inside any GUI with a text input widget, becomes easy. Heck, even Idle could use it rather than a socket for its interactive interpreter. ---------------------------------------------------------------------- Comment By: Dave Schuyler (parameter) Date: 2005-04-28 16:38 Message: Logged In: YES user_id=473331 More control of sub-processes would be great. Several times in the past few years I've done stuff with os.system, popenN, or spawn* and it would be useful to have even more cross-platform consistency and support in this area. Anyway, this is just a vote to encurage other developers. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191964&group_id=5470 From noreply at sourceforge.net Fri Apr 29 04:31:00 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 04:31:02 2005 Subject: [ python-Feature Requests-1191697 ] slice indices different than integers Message-ID: Feature Requests item #1191697, was opened at 2005-04-28 08:42 Message generated for change (Comment added) made by cxdunn You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191697&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sebastien de Menten (sdementen) Assigned to: Nobody/Anonymous (nobody) Summary: slice indices different than integers Initial Comment: Hi, Slice notation is quite convenient while addressing some item in a collection. It would be nice to extend this notation by enabling any object instead of integer in slices. Example 1: let time_serie be an object representing a time serie and date and days object that manages dates and concept of days interval, one could indexed time_serie with: time_serie[date(2004,3,4):date(2004,5,3)] or time_serie[date(2004,3,4):date(2004,5,3):days(5)] Example 2: Let f be a numerical function with multiple arguments, one could get an array of results by using f[3:10:2, 1:5:3] or naturally :-) f[3.2:10.1:0.4, 1:5:3] Well, I think it is a matter of removing the check on argument of slices to enable the syntactic sugar start:end:step in __getitem__ calls as well as adding integer checks on slice attributes when using it in old way. ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-28 21:31 Message: Logged In: YES user_id=1267419 This already works. See page 95 of Python in a Nutshell, or check the docs. Slice objects are created when you specify a slice. They can contain anything. Special methods (like __getitem__ and __delitem__) are passed that object, so your own class could define those methonds and do anything you want with the arbitrary slice object. For example: 

class Ruler(object):
    def __getitem__(self, s):
        return s.stop - s.start

x = Ruler()
print x[5:7]
print x[4.2:5.1]
produces 2 0.9 -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191697&group_id=5470 From noreply at sourceforge.net Fri Apr 29 04:54:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 04:54:51 2005 Subject: [ python-Bugs-1191104 ] Warning ``error`` filter action is ignored. Message-ID: Bugs item #1191104, was opened at 2005-04-27 10:55 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ivan Vilata i Balaguer (ivilata) >Assigned to: Vinay Sajip (vsajip) Summary: Warning ``error`` filter action is ignored. Initial Comment: Hi all. It seems that setting the ``error`` action on a warning message once it has already been triggered does not allow it to be triggered again. As usual, the code is clearer than the explanation:: import warnings def do_warn(): warnings.warn("A warning.", DeprecationWarning) do_warn() warnings.filterwarnings('error', category=DeprecationWarning) do_warn() warnings.filterwarnings('always', category=DeprecationWarning) do_warn() The output of this program is:: nowarn.py:4: DeprecationWarning: A warning. warnings.warn("A warning.", DeprecationWarning) There is no exception raised, though the filter was instructed to turn the warning into an error. Take note that using ``always`` has similar results. Does it work in that way on purpose? I find it quite counterintuitive, IMHO. If this behaviour is intended, what would be the way to turn the second warning into an exception? Thanks! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 From noreply at sourceforge.net Fri Apr 29 10:17:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 10:17:34 2005 Subject: [ python-Bugs-1191104 ] Warning ``error`` filter action is ignored. Message-ID: Bugs item #1191104, was opened at 2005-04-27 15:55 Message generated for change (Comment added) made by vsajip You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Ivan Vilata i Balaguer (ivilata) Assigned to: Vinay Sajip (vsajip) Summary: Warning ``error`` filter action is ignored. Initial Comment: Hi all. It seems that setting the ``error`` action on a warning message once it has already been triggered does not allow it to be triggered again. As usual, the code is clearer than the explanation:: import warnings def do_warn(): warnings.warn("A warning.", DeprecationWarning) do_warn() warnings.filterwarnings('error', category=DeprecationWarning) do_warn() warnings.filterwarnings('always', category=DeprecationWarning) do_warn() The output of this program is:: nowarn.py:4: DeprecationWarning: A warning. warnings.warn("A warning.", DeprecationWarning) There is no exception raised, though the filter was instructed to turn the warning into an error. Take note that using ``always`` has similar results. Does it work in that way on purpose? I find it quite counterintuitive, IMHO. If this behaviour is intended, what would be the way to turn the second warning into an exception? Thanks! ---------------------------------------------------------------------- >Comment By: Vinay Sajip (vsajip) Date: 2005-04-29 08:17 Message: Logged In: YES user_id=308438 This does not appear to be a bug. For example, the following script (indentation may get messed up): #-------------------------------------- import sys, warnings def do_warn(): fn = sys.modules["__main__"].__file__ warnings.warn_explicit("A warning.", DeprecationWarning, fn, 0) def main(): do_warn() warnings.filterwarnings('error', category=DeprecationWarning) try: do_warn() except Exception, e: print "exception handled: %s" % e warnings.filterwarnings('always', category=DeprecationWarning) do_warn() if __name__ == "__main__": main() #-------------------------------------- prints C:\temp\nowarn.py:0: DeprecationWarning: A warning. exception handled: A warning. C:\temp\nowarn.py:0: DeprecationWarning: A warning. So the problem is that if you want explicit warnings, you need to use warn_explicit() rather than warn(). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 From noreply at sourceforge.net Fri Apr 29 11:53:05 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 11:53:08 2005 Subject: [ python-Feature Requests-1191697 ] slice indices different than integers Message-ID: Feature Requests item #1191697, was opened at 2005-04-28 14:42 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191697&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Sebastien de Menten (sdementen) >Assigned to: Michael Hudson (mwh) Summary: slice indices different than integers Initial Comment: Hi, Slice notation is quite convenient while addressing some item in a collection. It would be nice to extend this notation by enabling any object instead of integer in slices. Example 1: let time_serie be an object representing a time serie and date and days object that manages dates and concept of days interval, one could indexed time_serie with: time_serie[date(2004,3,4):date(2004,5,3)] or time_serie[date(2004,3,4):date(2004,5,3):days(5)] Example 2: Let f be a numerical function with multiple arguments, one could get an array of results by using f[3:10:2, 1:5:3] or naturally :-) f[3.2:10.1:0.4, 1:5:3] Well, I think it is a matter of removing the check on argument of slices to enable the syntactic sugar start:end:step in __getitem__ calls as well as adding integer checks on slice attributes when using it in old way. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2005-04-29 10:53 Message: Logged In: YES user_id=6656 Closing, because you can do this already: >>> class D(object): ... def __getitem__(self, item): ... print item.start, item.stop, item.step ... >>> import datetime >>> D()[datetime.datetime.now():1.3:D] 2005-04-29 10:56:17.368061 1.3 ---------------------------------------------------------------------- Comment By: Christopher Dunn (cxdunn) Date: 2005-04-29 03:31 Message: Logged In: YES user_id=1267419 This already works. See page 95 of Python in a Nutshell, or check the docs. Slice objects are created when you specify a slice. They can contain anything. Special methods (like __getitem__ and __delitem__) are passed that object, so your own class could define those methonds and do anything you want with the arbitrary slice object. For example: 

class Ruler(object):
    def __getitem__(self, s):
        return s.stop - s.start

x = Ruler()
print x[5:7]
print x[4.2:5.1]
produces 2 0.9 -cxdunn ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191697&group_id=5470 From noreply at sourceforge.net Fri Apr 29 12:30:56 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 12:30:58 2005 Subject: [ python-Bugs-1192315 ] 'clear -1' in pdb Message-ID: Bugs item #1192315, was opened at 2005-04-29 13:30 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192315&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Pechkinzzz (mpech) Assigned to: Nobody/Anonymous (nobody) Summary: 'clear -1' in pdb Initial Comment: Delete breakpoints like in %subj% is great feature, but wrong. Add additional check like in do_enable() if not (0 <= i < len(bdb.Breakpoint.bpbynumber)): ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192315&group_id=5470 From noreply at sourceforge.net Fri Apr 29 12:54:46 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 12:54:49 2005 Subject: [ python-Bugs-1191104 ] Warning ``error`` filter action is ignored. Message-ID: Bugs item #1191104, was opened at 2005-04-27 17:55 Message generated for change (Comment added) made by ivilata You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Invalid Priority: 5 Submitted By: Ivan Vilata i Balaguer (ivilata) Assigned to: Vinay Sajip (vsajip) Summary: Warning ``error`` filter action is ignored. Initial Comment: Hi all. It seems that setting the ``error`` action on a warning message once it has already been triggered does not allow it to be triggered again. As usual, the code is clearer than the explanation:: import warnings def do_warn(): warnings.warn("A warning.", DeprecationWarning) do_warn() warnings.filterwarnings('error', category=DeprecationWarning) do_warn() warnings.filterwarnings('always', category=DeprecationWarning) do_warn() The output of this program is:: nowarn.py:4: DeprecationWarning: A warning. warnings.warn("A warning.", DeprecationWarning) There is no exception raised, though the filter was instructed to turn the warning into an error. Take note that using ``always`` has similar results. Does it work in that way on purpose? I find it quite counterintuitive, IMHO. If this behaviour is intended, what would be the way to turn the second warning into an exception? Thanks! ---------------------------------------------------------------------- >Comment By: Ivan Vilata i Balaguer (ivilata) Date: 2005-04-29 12:54 Message: Logged In: YES user_id=1064183 OK, I see. Then the documentation for ``warnings.warn()`` may be more precise. Where it says ?This function raises an exception if the particular warning issued is changed into an error by the warnings filter see above.? it may append ?and that warning has not already been filtered?. Anyway, I still think that one should expect to *always* get an exception once the warnings filter has been instructed to turn a warning into an exception, instead of maybe filtering it. The later could lead to some errors to pass unadvertised and get delayed. It also disables the possibility of turning every warning into an error in the middle of a session (? la gcc's ``-pedantic-errors`` option). My proposal is not to filter a warning once it has been instructed to cause an exception. The existing code should not need to worry about using ``warn()`` or ``warn_explicit()`` (since in the usual situation, duplicate warnings --not exceptions-- should still be filtered). Thanks for your attention, Ivan ---------------------------------------------------------------------- Comment By: Vinay Sajip (vsajip) Date: 2005-04-29 10:17 Message: Logged In: YES user_id=308438 This does not appear to be a bug. For example, the following script (indentation may get messed up): #-------------------------------------- import sys, warnings def do_warn(): fn = sys.modules["__main__"].__file__ warnings.warn_explicit("A warning.", DeprecationWarning, fn, 0) def main(): do_warn() warnings.filterwarnings('error', category=DeprecationWarning) try: do_warn() except Exception, e: print "exception handled: %s" % e warnings.filterwarnings('always', category=DeprecationWarning) do_warn() if __name__ == "__main__": main() #-------------------------------------- prints C:\temp\nowarn.py:0: DeprecationWarning: A warning. exception handled: A warning. C:\temp\nowarn.py:0: DeprecationWarning: A warning. So the problem is that if you want explicit warnings, you need to use warn_explicit() rather than warn(). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1191104&group_id=5470 From noreply at sourceforge.net Fri Apr 29 15:03:15 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 15:03:18 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 19:05 Message generated for change (Comment added) made by poromenos You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None >Status: Open Resolution: Rejected Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Poromenos (poromenos) Date: 2005-04-29 16:03 Message: Logged In: YES user_id=267121 There was an oversight on my part... Translate can only be used to change individual characters, what I am proposing could replace strings of multiple characters, essentially concatenating multiple replace()s in one: >>> "h.w".replace(["h", ".", "w"], ["Hello", " ", "World!"]) 'Hello, World!' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-19 01:23 Message: Logged In: YES user_id=267121 Ah, I did not know that... The docs are a bit complicated on . translate, but since it can do that, yes, it would be unwise to implement my suggestion. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 01:20 Message: Logged In: YES user_id=80475 I'm -1 on complicating str.replace()'s API for functionality that substantially duplicates that offered by str.translate(): >>> "Hello world".translate(string.maketrans('ed', 'ae')) 'Hallo worle' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-19 00:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 00:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Fri Apr 29 15:38:14 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 15:38:17 2005 Subject: [ python-Bugs-1189811 ] pydoc may hide non-private doc strings. Message-ID: Bugs item #1189811, was opened at 2005-04-25 16:00 Message generated for change (Comment added) made by jlivingston You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189811&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: J Livingston (jlivingston) Assigned to: Nobody/Anonymous (nobody) Summary: pydoc may hide non-private doc strings. Initial Comment: I am using Python version 2.3.4, pydoc version 1.86.8.1 on WinXP SP2. pydoc's visiblename() method indicates "Private names are hidden, but special names are displayed". However, visiblename's private name qualifier seems to be (name.startswith('_')) while Python's private name qualifier is something more along the lines of ((name.startswith('__') and ((name[-1] != '_') or ((name[-1] == '_') and (name[-2] != '_')))). Having said that, it would be useful if a command line switch enabled documentation for private names. This would be helpful in a development environment... ---------------------------------------------------------------------- >Comment By: J Livingston (jlivingston) Date: 2005-04-29 08:38 Message: Logged In: YES user_id=1160595 An amendment is need for this enhancement request... The focus at the time of submitting the request was that of classes and their private data. It seems the comment in the code and the code itself may have been written with the focus of modules and the rule that prohibits importing names that begin with '_' when an 'import *' is used for a module. This may open another can of worms altogether. To distinguish the 'private' names of modules and classes visiblenames() would have to employ different logic for each. Also, one would wonder if the elements of an __all__ list would have to be considered the non-private names of a module or package... From this point, a discussion of what is considered ?private? could become quite lengthy and also subjective. Although discussion regarding this level of complexity may be relevant at some point, the intent of the enhancement request was primarily to allow 'private' doc strings to be visible if explicitly requested. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1189811&group_id=5470 From noreply at sourceforge.net Fri Apr 29 16:10:25 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 16:10:28 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 11:05 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: Rejected Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-29 09:10 Message: Logged In: YES user_id=80475 Given the multiple alternative input matches, this is a job for regular expressions. See the string.substitute() source code for an approach to making the transformation you outlined. I do not think multi-replace is sufficiently common to warrant a change to the API. If needed and if the regexp solution is too hard, then a regular for-loop is a reasonable alternative: for old, new in zip(oldlist, newlist): s = s.replace(old, new) ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-29 08:03 Message: Logged In: YES user_id=267121 There was an oversight on my part... Translate can only be used to change individual characters, what I am proposing could replace strings of multiple characters, essentially concatenating multiple replace()s in one: >>> "h.w".replace(["h", ".", "w"], ["Hello", " ", "World!"]) 'Hello, World!' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-18 17:23 Message: Logged In: YES user_id=267121 Ah, I did not know that... The docs are a bit complicated on . translate, but since it can do that, yes, it would be unwise to implement my suggestion. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:20 Message: Logged In: YES user_id=80475 I'm -1 on complicating str.replace()'s API for functionality that substantially duplicates that offered by str.translate(): >>> "Hello world".translate(string.maketrans('ed', 'ae')) 'Hallo worle' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-18 16:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 16:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Fri Apr 29 16:32:42 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 16:32:46 2005 Subject: [ python-Feature Requests-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Feature Requests item #1190596, was opened at 2005-04-26 18:10 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) >Assigned to: Nobody/Anonymous (nobody) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-29 10:32 Message: Logged In: YES user_id=31435 +1 from me. It's a very simple change, allowing callers to break dependence on a one-size-doesn't-fit-all global in a clear and thread-safe way. Doesn't break any existing code. Tiny slowdown in functions that aren't speed-critical anyway. The patch needs doc and test suite changes too, but what's here is fine by me so far as it goes. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 18:51 Message: Logged In: YES user_id=80475 Tim, I'm -0 on this one. Do you have an opinion? Essentially, the OP wants an alternate, thread-safe interface to specifying firstweekday on a per-call basis. It can already be done with a wrapper function and a lock, so it is just a convenience issue. ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 09:02 Message: Logged In: YES user_id=127625 WIth the patch, if you pass 'firstweekday' when calling 'monthcalendar' or 'weekheader', there is no race, and therefore no need to guard the global, because the global is ignored. Nothing else in the module depends on the value of that global except those two functions. I'm attaching a doctest which demonstrates the use case. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 08:36 Message: Logged In: YES user_id=80475 Write a simple wrapper to save and restore the global state: >>> def mymonthcalendar(year, month, first=None): if first is None: return monthcalendar(year, month) saved = firstweekday() setfirstweekday(first) rows = monthcalendar(year, month) setfirstweekday(saved) return rows That is essentially what the existing API is for. For a multi-threaded environment, you would need to add locks to th e critical section in the above code (the same would be true for your proposed patch). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 08:24 Message: Logged In: YES user_id=127625 Different users of a long-running Python process may have different preferences for the first weekday; if the application tries to use the setfirstweekday() API for each of them, then they end up racing to set it. Making the option selectable per-call makes it side-effect free, and therefore more robust for multi-user applications. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 From noreply at sourceforge.net Fri Apr 29 16:44:45 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 16:44:47 2005 Subject: [ python-Feature Requests-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Feature Requests item #1190596, was opened at 2005-04-26 17:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) >Assigned to: Raymond Hettinger (rhettinger) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-29 09:44 Message: Logged In: YES user_id=80475 Okay, I've got it from here. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-29 09:32 Message: Logged In: YES user_id=31435 +1 from me. It's a very simple change, allowing callers to break dependence on a one-size-doesn't-fit-all global in a clear and thread-safe way. Doesn't break any existing code. Tiny slowdown in functions that aren't speed-critical anyway. The patch needs doc and test suite changes too, but what's here is fine by me so far as it goes. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 17:51 Message: Logged In: YES user_id=80475 Tim, I'm -0 on this one. Do you have an opinion? Essentially, the OP wants an alternate, thread-safe interface to specifying firstweekday on a per-call basis. It can already be done with a wrapper function and a lock, so it is just a convenience issue. ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 08:02 Message: Logged In: YES user_id=127625 WIth the patch, if you pass 'firstweekday' when calling 'monthcalendar' or 'weekheader', there is no race, and therefore no need to guard the global, because the global is ignored. Nothing else in the module depends on the value of that global except those two functions. I'm attaching a doctest which demonstrates the use case. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:36 Message: Logged In: YES user_id=80475 Write a simple wrapper to save and restore the global state: >>> def mymonthcalendar(year, month, first=None): if first is None: return monthcalendar(year, month) saved = firstweekday() setfirstweekday(first) rows = monthcalendar(year, month) setfirstweekday(saved) return rows That is essentially what the existing API is for. For a multi-threaded environment, you would need to add locks to th e critical section in the above code (the same would be true for your proposed patch). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:24 Message: Logged In: YES user_id=127625 Different users of a long-running Python process may have different preferences for the first weekday; if the application tries to use the setfirstweekday() API for each of them, then they end up racing to set it. Making the option selectable per-call makes it side-effect free, and therefore more robust for multi-user applications. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 06:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 06:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 01:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 From noreply at sourceforge.net Fri Apr 29 17:04:47 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 17:04:51 2005 Subject: [ python-Feature Requests-1190596 ] calendar._firstweekday is too hard-wired Message-ID: Feature Requests item #1190596, was opened at 2005-04-26 18:10 Message generated for change (Comment added) made by tseaver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tres Seaver (tseaver) Assigned to: Raymond Hettinger (rhettinger) Summary: calendar._firstweekday is too hard-wired Initial Comment: Long-running applications which need to make use of the 'calendar' module may need to present calendars to different users using the users' preferred settings. Currently, the two functions which rely on the locale-specific setting are too inflexible: they assume that the module-level global should always govern. The attached patch adds defaulted arguments to these functions, and uses the module-level global only where the caller does not pass in a value. ---------------------------------------------------------------------- >Comment By: Tres Seaver (tseaver) Date: 2005-04-29 11:04 Message: Logged In: YES user_id=127625 I somehow dropped the test patch on the floor. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-29 10:44 Message: Logged In: YES user_id=80475 Okay, I've got it from here. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2005-04-29 10:32 Message: Logged In: YES user_id=31435 +1 from me. It's a very simple change, allowing callers to break dependence on a one-size-doesn't-fit-all global in a clear and thread-safe way. Doesn't break any existing code. Tiny slowdown in functions that aren't speed-critical anyway. The patch needs doc and test suite changes too, but what's here is fine by me so far as it goes. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 18:51 Message: Logged In: YES user_id=80475 Tim, I'm -0 on this one. Do you have an opinion? Essentially, the OP wants an alternate, thread-safe interface to specifying firstweekday on a per-call basis. It can already be done with a wrapper function and a lock, so it is just a convenience issue. ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 09:02 Message: Logged In: YES user_id=127625 WIth the patch, if you pass 'firstweekday' when calling 'monthcalendar' or 'weekheader', there is no race, and therefore no need to guard the global, because the global is ignored. Nothing else in the module depends on the value of that global except those two functions. I'm attaching a doctest which demonstrates the use case. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 08:36 Message: Logged In: YES user_id=80475 Write a simple wrapper to save and restore the global state: >>> def mymonthcalendar(year, month, first=None): if first is None: return monthcalendar(year, month) saved = firstweekday() setfirstweekday(first) rows = monthcalendar(year, month) setfirstweekday(saved) return rows That is essentially what the existing API is for. For a multi-threaded environment, you would need to add locks to th e critical section in the above code (the same would be true for your proposed patch). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 08:24 Message: Logged In: YES user_id=127625 Different users of a long-running Python process may have different preferences for the first weekday; if the application tries to use the setfirstweekday() API for each of them, then they end up racing to set it. Making the option selectable per-call makes it side-effect free, and therefore more robust for multi-user applications. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 07:42 Message: Logged In: YES user_id=80475 I do not follow what can be done with your patch that cannot currently be done with setfirstweekday(). ---------------------------------------------------------------------- Comment By: Tres Seaver (tseaver) Date: 2005-04-28 07:06 Message: Logged In: YES user_id=127625 Aarrgh, what is the point of that silly checkbox, anyway? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-28 02:59 Message: Logged In: YES user_id=80475 The attachment didn't make it. Please try again and make sure to check-off the upload box. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1190596&group_id=5470 From noreply at sourceforge.net Fri Apr 29 19:36:31 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 19:36:33 2005 Subject: [ python-Bugs-1192554 ] doctest's ELLIPSIS and multiline statements Message-ID: Bugs item #1192554, was opened at 2005-04-29 19:36 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192554&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: S?bastien Boisg?rault (boisgerault) Assigned to: Nobody/Anonymous (nobody) Summary: doctest's ELLIPSIS and multiline statements Initial Comment: The doctest ELLPSIS marker (default: "...") may be confused by the doctest parser with the multiline statement marker ("..."). Example: in the following code, the intent was to accept any result after "print 42". This is NOT a multiline statement, but however the test fails (Expected: nothing, Got: 42). ---------------------------------------- #!/usr/bin/env python import doctest def test(): """ >>> print 42 #doctest: +ELLIPSIS ... """ def run(): "Run the test." doctest.testmod() if __name__ == '__main__': run() ---------------------------------------- ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192554&group_id=5470 From noreply at sourceforge.net Fri Apr 29 19:52:43 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Apr 29 19:52:46 2005 Subject: [ python-Bugs-1192554 ] doctest's ELLIPSIS and multiline statements Message-ID: Bugs item #1192554, was opened at 2005-04-29 13:36 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192554&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: S?bastien Boisg?rault (boisgerault) Assigned to: Nobody/Anonymous (nobody) Summary: doctest's ELLIPSIS and multiline statements Initial Comment: The doctest ELLPSIS marker (default: "...") may be confused by the doctest parser with the multiline statement marker ("..."). Example: in the following code, the intent was to accept any result after "print 42". This is NOT a multiline statement, but however the test fails (Expected: nothing, Got: 42). ---------------------------------------- #!/usr/bin/env python import doctest def test(): """ >>> print 42 #doctest: +ELLIPSIS ... """ def run(): "Run the test." doctest.testmod() if __name__ == '__main__': run() ---------------------------------------- ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2005-04-29 13:52 Message: Logged In: YES user_id=31435 That's true. doctest has few syntax requirements, but the inability to start an expected output block with "..." has always been one of them, and is independent of the ELLIPSIS gimmick. I doubt this will change, in part because the complications needed to "do something about it" are probably pig ugly, in part because it's so rare a desire, and in part because there are easy ways to work around it (like arranging for the expected output to start with something other than '...'). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192554&group_id=5470 From noreply at sourceforge.net Sat Apr 30 00:04:58 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 00:05:01 2005 Subject: [ python-Bugs-999042 ] Compiler module doesn't handle global statement correctly Message-ID: Bugs item #999042, was opened at 2004-07-27 22:15 Message generated for change (Comment added) made by dsuch You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=999042&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Jim Fulton (dcjim) Assigned to: Nobody/Anonymous (nobody) Summary: Compiler module doesn't handle global statement correctly Initial Comment: If we don't use the compiler module: >>> code = 'global x\nx=1' >>> d1={'__builtins__': {}}; d2={}; exec code in d1, d2 >>> d1, d2 ({'__builtins__': {}, 'x': 1}, {}) with the compiler module: >>> code = compiler.compile('global x\nx=1', 'd', 'exec') >>> d1={'__builtins__': {}}; d2={}; exec code in d1, d2 >>> d1, d2 ({'__builtins__': {}}, {'x': 1}) global is ignored ---------------------------------------------------------------------- Comment By: Darek Suchojad (dsuch) Date: 2005-04-29 22:04 Message: Logged In: YES user_id=954779 Hi, I have submitted a simple fix some time ago python.org/sf/1090482, do you think it is correct? ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2004-07-28 14:01 Message: Logged In: YES user_id=73023 Also in 2.3 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=999042&group_id=5470 From noreply at sourceforge.net Sat Apr 30 00:51:48 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 00:51:51 2005 Subject: [ python-Bugs-1192777 ] docstring error Message-ID: Bugs item #1192777, was opened at 2005-04-29 17:51 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192777&group_id=5470 Category: Python Library Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Christopher Smith (smichr) Assigned to: Nobody/Anonymous (nobody) Summary: docstring error Initial Comment: In 2.4/Lib/test/test_binop.py, two double quotes are missing at the beginning of the following function's docstring: def __rdivmod__(self, other): "Divide two Rats, returning quotient and remainder (reversed args).""" (I know it's minor, but you said "keep it under your pillow" ;-) Actually. I found it while testing out a comment stripping program that uses the tokenize module. /c ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192777&group_id=5470 From noreply at sourceforge.net Sat Apr 30 07:53:12 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 07:53:15 2005 Subject: [ python-Bugs-1192777 ] docstring error Message-ID: Bugs item #1192777, was opened at 2005-04-29 15:51 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192777&group_id=5470 Category: Python Library Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Christopher Smith (smichr) Assigned to: Nobody/Anonymous (nobody) Summary: docstring error Initial Comment: In 2.4/Lib/test/test_binop.py, two double quotes are missing at the beginning of the following function's docstring: def __rdivmod__(self, other): "Divide two Rats, returning quotient and remainder (reversed args).""" (I know it's minor, but you said "keep it under your pillow" ;-) Actually. I found it while testing out a comment stripping program that uses the tokenize module. /c ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2005-04-29 22:53 Message: Logged In: YES user_id=357491 Fixed in rev. 1.8 for 2.5 and rev. 1.7.20.1 for 2.4 . Thanks, Christopher. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1192777&group_id=5470 From noreply at sourceforge.net Sat Apr 30 11:48:54 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 11:48:57 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 19:05 Message generated for change (Comment added) made by poromenos You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: Rejected Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Poromenos (poromenos) Date: 2005-04-30 12:48 Message: Logged In: YES user_id=267121 That is true, the alternative loop is quite usable, but the API change would be backwards-compatible, and the implementation is not very difficult... I just see this as a nice feature of replace, it's not really necessary if it'll break other stuff. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-29 17:10 Message: Logged In: YES user_id=80475 Given the multiple alternative input matches, this is a job for regular expressions. See the string.substitute() source code for an approach to making the transformation you outlined. I do not think multi-replace is sufficiently common to warrant a change to the API. If needed and if the regexp solution is too hard, then a regular for-loop is a reasonable alternative: for old, new in zip(oldlist, newlist): s = s.replace(old, new) ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-29 16:03 Message: Logged In: YES user_id=267121 There was an oversight on my part... Translate can only be used to change individual characters, what I am proposing could replace strings of multiple characters, essentially concatenating multiple replace()s in one: >>> "h.w".replace(["h", ".", "w"], ["Hello", " ", "World!"]) 'Hello, World!' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-19 01:23 Message: Logged In: YES user_id=267121 Ah, I did not know that... The docs are a bit complicated on . translate, but since it can do that, yes, it would be unwise to implement my suggestion. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 01:20 Message: Logged In: YES user_id=80475 I'm -1 on complicating str.replace()'s API for functionality that substantially duplicates that offered by str.translate(): >>> "Hello world".translate(string.maketrans('ed', 'ae')) 'Hallo worle' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-19 00:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-19 00:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Sat Apr 30 14:30:59 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 14:31:02 2005 Subject: [ python-Feature Requests-1184678 ] "replace" function should accept lists. Message-ID: Feature Requests item #1184678, was opened at 2005-04-17 11:05 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 Category: None Group: None Status: Open Resolution: Rejected Priority: 5 Submitted By: Poromenos (poromenos) Assigned to: Nobody/Anonymous (nobody) Summary: "replace" function should accept lists. Initial Comment: It would be nice if the "replace" function accepted lists/ tuples and replaced each item in the "old" list with the corresponding item in the "new" list. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-30 07:30 Message: Logged In: YES user_id=80475 Sorry, am not going to gum up the API for this. It doesn't come up much and when it does it is an easy problem. Each additional option on a function makes it harder to learn, use, and remember. Likewise, it complicates maintenance. Also, this one has its own complexities that make it worth avoiding (examples can be contrived when the for-loop version produces a different result than replacing each match as found). A good implementation would take time. It would involve regexps and have to be done for unicode objects. ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-30 04:48 Message: Logged In: YES user_id=267121 That is true, the alternative loop is quite usable, but the API change would be backwards-compatible, and the implementation is not very difficult... I just see this as a nice feature of replace, it's not really necessary if it'll break other stuff. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-29 09:10 Message: Logged In: YES user_id=80475 Given the multiple alternative input matches, this is a job for regular expressions. See the string.substitute() source code for an approach to making the transformation you outlined. I do not think multi-replace is sufficiently common to warrant a change to the API. If needed and if the regexp solution is too hard, then a regular for-loop is a reasonable alternative: for old, new in zip(oldlist, newlist): s = s.replace(old, new) ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-29 08:03 Message: Logged In: YES user_id=267121 There was an oversight on my part... Translate can only be used to change individual characters, what I am proposing could replace strings of multiple characters, essentially concatenating multiple replace()s in one: >>> "h.w".replace(["h", ".", "w"], ["Hello", " ", "World!"]) 'Hello, World!' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-18 17:23 Message: Logged In: YES user_id=267121 Ah, I did not know that... The docs are a bit complicated on . translate, but since it can do that, yes, it would be unwise to implement my suggestion. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 17:20 Message: Logged In: YES user_id=80475 I'm -1 on complicating str.replace()'s API for functionality that substantially duplicates that offered by str.translate(): >>> "Hello world".translate(string.maketrans('ed', 'ae')) 'Hallo worle' ---------------------------------------------------------------------- Comment By: Poromenos (poromenos) Date: 2005-04-18 16:15 Message: Logged In: YES user_id=267121 Hmm, actually I was requesting that string.replace() accepted lists of substitutions, like so: >>> "Hello world".replace(["e", "d"], ["a", "e"]) 'Hallo worle' But your suggestion would also be very useful. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-18 16:11 Message: Logged In: YES user_id=80475 Are you requesting that lists be given a replace() method that parallels the replace() method for strings? def replace(self, old, new): result = [] for elem in self: if elem == old: result.append(new) else: result.append(elem) self[:] = result ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1184678&group_id=5470 From noreply at sourceforge.net Sat Apr 30 14:32:21 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 14:32:24 2005 Subject: [ python-Feature Requests-1191699 ] make slices pickable Message-ID: Feature Requests item #1191699, was opened at 2005-04-28 08:44 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470 >Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sebastien de Menten (sdementen) Assigned to: Nobody/Anonymous (nobody) Summary: make slices pickable Initial Comment: As suggested by the summary, provide pickability of slices by default. Currently one has to use copy_reg module to register slices as pickables. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-30 07:32 Message: Logged In: YES user_id=80475 Use cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470 From noreply at sourceforge.net Sat Apr 30 16:01:32 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 16:01:35 2005 Subject: [ python-Bugs-1193001 ] Notation Message-ID: Bugs item #1193001, was opened at 2005-04-30 16:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1193001&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Mythril (mythrix) Assigned to: Nobody/Anonymous (nobody) Summary: Notation Initial Comment: In Notation, it says the BNF form used is: name: lc_letter (lc_letter | "_")* lc_letter: "a"..."z" But in the documentation, it is: identifier ::= (letter|"_") (letter | digit | "_")* letter ::= lowercase | uppercase (ie, ::=, and not :) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1193001&group_id=5470 From noreply at sourceforge.net Sat Apr 30 19:37:22 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 19:37:24 2005 Subject: [ python-Bugs-1193061 ] Python and Turkish Locale Message-ID: Bugs item #1193061, was opened at 2005-04-30 20:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1193061&group_id=5470 Category: Unicode Group: None Status: Open Resolution: None Priority: 5 Submitted By: S.?ağlar Onur (caglar) Assigned to: M.-A. Lemburg (lemburg) Summary: Python and Turkish Locale Initial Comment: On behalf of this thread; http://mail.python.org/pipermail/python-dev/2005-April/052968.html As described in http://www.i18nguy.com/unicode/turkish-i18n.html [ How Applications Fail With Turkish Language ] , Turkish has 4 "i" in their alphabet. Without --with-wctype-functions support Python convert these characters locare-independent manner in tr_TR.UTF-8 locale. So all conversitons maps to "i" or "I" which is wrong in Turkish locale. So if Python Developers will remove the wctype functions from Python, then there must be a locale-dependent upper/lower funtion to handle these characters properly. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1193061&group_id=5470 From noreply at sourceforge.net Sat Apr 30 20:02:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 20:02:51 2005 Subject: [ python-Feature Requests-1191699 ] make slices pickable Message-ID: Feature Requests item #1191699, was opened at 2005-04-28 13:44 Message generated for change (Comment added) made by sdementen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sebastien de Menten (sdementen) Assigned to: Nobody/Anonymous (nobody) Summary: make slices pickable Initial Comment: As suggested by the summary, provide pickability of slices by default. Currently one has to use copy_reg module to register slices as pickables. ---------------------------------------------------------------------- >Comment By: Sebastien de Menten (sdementen) Date: 2005-04-30 18:02 Message: Logged In: YES user_id=820079 Use case for pickable slices. Let A be a numeric array of size M x N. We want to consider sub-arrays in this array like A[:4, :] (the first 4 lines of the array). If we want to store both the array and the information of sub-arrays structures, we need to store slices (we can also store start/end indices of the array ... but this is call a slice so it would be better to have pickable slices). In fact, whenever one wants to write generic algorithm working on "sublist of lists" or "subarrays of arrays" or "sub-items of collection of items", it is nicer to use slice objects explicitly and so store them also explicitly. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-04-30 12:32 Message: Logged In: YES user_id=80475 Use cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1191699&group_id=5470 From noreply at sourceforge.net Sat Apr 30 21:03:35 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 21:03:37 2005 Subject: [ python-Bugs-1193099 ] Embedded python thread crashes Message-ID: Bugs item #1193099, was opened at 2005-04-30 12:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1193099&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: ugodiggi (ugodiggi) Assigned to: Nobody/Anonymous (nobody) Summary: Embedded python thread crashes Initial Comment: The following code crashes about 1/3 of the times. My platform is Python 2.4.1 on WXP (I tried the release version from the msi and the debug version built by me). I can reproduce the same behavior on another wxp system, under python 2.4. The crash happens (in the python thread) while the main thread is in Py_Finalize. I traced the crash to _Py_ForgetReference(op) in object.c at line 1847, where I have op->_ob_prev == NULL. The open file seems to be the issue, since if I remove all the references to the file I cannot get the program to crash. Cheers and ciao Ugo ////////////////////////// TestPyThreads.cpp ////////////////////////// #include // Sleep #include "Python.h" int main() { PyEval_InitThreads(); Py_Initialize(); PyGILState_STATE main_restore_state = PyGILState_UNLOCKED; PyGILState_Release(main_restore_state); // start the thread { PyGILState_STATE state = PyGILState_Ensure(); int trash = PyRun_SimpleString( "import thread\n" "import time\n" "def foo():\n" " f = open('pippo.out', 'w', 0)\n" " i = 0;\n" " while 1:\n" " f.write('%d\n'%i)\n" " time.sleep(0.01)\n" " i += 1\n" "t = thread.start_new_thread(foo, ())\n" ); PyGILState_Release(state); } // wait 300 ms Sleep(300); PyGILState_Ensure(); Py_Finalize(); return 0; } ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1193099&group_id=5470 From noreply at sourceforge.net Sat Apr 30 22:31:09 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 22:31:12 2005 Subject: [ python-Feature Requests-1193128 ] 'str'.translate(None) => identity translation Message-ID: Feature Requests item #1193128, was opened at 2005-04-30 13:31 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1193128&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Bengt Richter (bokr) Assigned to: Nobody/Anonymous (nobody) Summary: 'str'.translate(None) => identity translation Initial Comment: This feature would permit passing None as the first argument to str.translate in place of an identity translation table like ''.join([[chr(i) for i in xrange(256)]) This should be handy for deleting characters, e.g., as in s = s.translate(None, delchars) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1193128&group_id=5470 From noreply at sourceforge.net Sat Apr 30 22:34:49 2005 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Apr 30 22:34:51 2005 Subject: [ python-Feature Requests-1193128 ] 'str'.translate(None) => identity translation Message-ID: Feature Requests item #1193128, was opened at 2005-04-30 13:31 Message generated for change (Settings changed) made by bokr You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1193128&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Bengt Richter (bokr) >Assigned to: Raymond Hettinger (rhettinger) Summary: 'str'.translate(None) => identity translation Initial Comment: This feature would permit passing None as the first argument to str.translate in place of an identity translation table like ''.join([[chr(i) for i in xrange(256)]) This should be handy for deleting characters, e.g., as in s = s.translate(None, delchars) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=355470&aid=1193128&group_id=5470