From noreply at sourceforge.net Sun Apr 1 15:46:15 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 06:46:15 -0700 Subject: [Patches] [ python-Patches-1692335 ] Move initial args assignment to BaseException.__new__ Message-ID: Patches item #1692335, was opened at 2007-04-01 15:46 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692335&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: ?iga Seilnacht (zseil) Assigned to: Nobody/Anonymous (nobody) Summary: Move initial args assignment to BaseException.__new__ Initial Comment: Pickling exceptions fails when an Exception class requires an argument in the constructor, but doesn't call its base class' constructor. See this mail for details: http://mail.python.org/pipermail/python-dev/2007-April/072416.html This patch simply moves initial args assignment to BaseException.__new__. This should fix most of the problems, because it is very unlikely that an exception overwrites the __new__ method; exceptions used to be old style classes, which don't support the __new__ special method. The args attribute is still overwritten in all the __init__ methods, so there shouldn't be any backward compatibility problems. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692335&group_id=5470 From noreply at sourceforge.net Sun Apr 1 15:47:18 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 06:47:18 -0700 Subject: [Patches] [ python-Patches-1692335 ] Move initial args assignment to BaseException.__new__ Message-ID: Patches item #1692335, was opened at 2007-04-01 15:46 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692335&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: ?iga Seilnacht (zseil) Assigned to: Nobody/Anonymous (nobody) Summary: Move initial args assignment to BaseException.__new__ Initial Comment: Pickling exceptions fails when an Exception class requires an argument in the constructor, but doesn't call its base class' constructor. See this mail for details: http://mail.python.org/pipermail/python-dev/2007-April/072416.html This patch simply moves initial args assignment to BaseException.__new__. This should fix most of the problems, because it is very unlikely that an exception overwrites the __new__ method; exceptions used to be old style classes, which don't support the __new__ special method. The args attribute is still overwritten in all the __init__ methods, so there shouldn't be any backward compatibility problems. ---------------------------------------------------------------------- >Comment By: ?iga Seilnacht (zseil) Date: 2007-04-01 15:47 Message: Logged In: YES user_id=1326842 Originator: YES File Added: exc_args_25.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692335&group_id=5470 From noreply at sourceforge.net Sun Apr 1 20:00:51 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 11:00:51 -0700 Subject: [Patches] [ python-Patches-1691070 ] Fix for bug #1283289 Message-ID: Patches item #1691070, was opened at 2007-03-30 06:45 Message generated for change (Comment added) made by paulhankin You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Roger Upole (rupole) Assigned to: Nobody/Anonymous (nobody) Summary: Fix for bug #1283289 Initial Comment: This is a fix for [ 1283289 ] PyArg_ParseTupleAndKeywords gives misleading error message. It also yields a 10-15% decrease in cpu usage, depending on the number of arguments. ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 18:00 Message: Logged In: YES user_id=1740099 Originator: NO Patched code compiles and passes test suite, and looks cleaner than the code it replaces. It works on the cases the bug report. There should be unit tests added to Lib/test/test_getargs2; I've skimmed the tests in there and can't see any that test combinations of kw, tuple and positional arguments. The patch replaces significant chunks of arg parsing code, so I'd be more convinced of its correctness if there were some unit tests. Certainly there should be at least one test for the bug the patch fixes. The code is 99% written in the prevailing style, but: Source contains several over-long lines, and non-standard comments, ie you should use /* XXX something .. */ and not /* ??? something .. ??? */ Also there are a few minor spacing inconsistencies which would be nice to tidy up. The diff is one level too high up - it contains the directory Python2.5/ It applies cleanly to the trunk though, it just makes it slightly more difficult to apply the patch. The original bug report requests a post to python-dev describing the new error messages. I suggest this post includes all cases where the new code produces a different message than the old. I suggest to proceed: 1. Add unit tests, both for the bug-fix and for existing functionality which has been modified 2. Fix up comments and long lines 3. Check with python-dev about new error messages ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 From noreply at sourceforge.net Sun Apr 1 20:30:05 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 11:30:05 -0700 Subject: [Patches] [ python-Patches-1685563 ] MSVCCompiler creates redundant and long PATH strings Message-ID: Patches item #1685563, was opened at 2007-03-21 15:05 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685563&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils and setup.py >Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Scott Dial (geekmug) >Assigned to: Neal Norwitz (nnorwitz) Summary: MSVCCompiler creates redundant and long PATH strings Initial Comment: The current code in MSVCCompiler.initialiaze will throw an OSError on the assignment to os.environ['path'] after repeated usages of MSVCCompilers because the string produced may exceeed 4096 characters (usually after several compilations). In the call to initialize, the needed paths for compiling via the compiler selected are added on to the front of the path regardless of what the current path is set to be. This is bad because we could be adding redundant paths (and thus exceeding the 4096 character limit needlessly) Furthmore, this happens regularly whenever anyone uses the VCToolkit2003 compiler because you end up in the "if self.__version >= 7:" control path of get_msvc_paths which merely returns the current path back to the callee. This results in initialize /duplicating the entire path/. The proposed patch adds a pass through the new path that removes duplicates while maintaining the ordering. Unfortunately it is difficult for me to provide a real test case for this as it arises out of trying to run the translate code with PyPy. However, I hope that the change is obviously fixing a bug that I've made clear exists. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-01 11:30 Message: Logged In: YES user_id=33168 Originator: NO Committed revision 54644. Committed revision 54645. (2.5) Backporting might be considered a little sketchy, but it fixes a real problem and is not too likely to break something important (ha! that's just inviting Murphy in). Please test 2.5.1c1 that will be coming out in a few days. I modified the patch to use a utility function and added comments and docstrings. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685563&group_id=5470 From noreply at sourceforge.net Sun Apr 1 20:58:16 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 11:58:16 -0700 Subject: [Patches] [ python-Patches-1685563 ] MSVCCompiler creates redundant and long PATH strings Message-ID: Patches item #1685563, was opened at 2007-03-21 18:05 Message generated for change (Comment added) made by geekmug You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685563&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils and setup.py Group: Python 2.5 Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Scott Dial (geekmug) Assigned to: Neal Norwitz (nnorwitz) Summary: MSVCCompiler creates redundant and long PATH strings Initial Comment: The current code in MSVCCompiler.initialiaze will throw an OSError on the assignment to os.environ['path'] after repeated usages of MSVCCompilers because the string produced may exceeed 4096 characters (usually after several compilations). In the call to initialize, the needed paths for compiling via the compiler selected are added on to the front of the path regardless of what the current path is set to be. This is bad because we could be adding redundant paths (and thus exceeding the 4096 character limit needlessly) Furthmore, this happens regularly whenever anyone uses the VCToolkit2003 compiler because you end up in the "if self.__version >= 7:" control path of get_msvc_paths which merely returns the current path back to the callee. This results in initialize /duplicating the entire path/. The proposed patch adds a pass through the new path that removes duplicates while maintaining the ordering. Unfortunately it is difficult for me to provide a real test case for this as it arises out of trying to run the translate code with PyPy. However, I hope that the change is obviously fixing a bug that I've made clear exists. ---------------------------------------------------------------------- >Comment By: Scott Dial (geekmug) Date: 2007-04-01 14:58 Message: Logged In: YES user_id=383208 Originator: YES The revised patch looks good. I think the comment about it being O(n**2) is not to be worrisome. I suppose you could argue for something like: return list(set([os.path.normpath(p) for p in paths])) However, you would lose the ordering, which must be maintained. At worst, the path can only be 4096 characters (at most 1024 entries of "C:\\"). Except in the face of what will be an error (the final path is going to be too large), we are bounded from above at n=2048, which is plenty fast. I think you probably guess that much, but at least my rationale for such a simple algorithm is recorded. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-01 14:30 Message: Logged In: YES user_id=33168 Originator: NO Committed revision 54644. Committed revision 54645. (2.5) Backporting might be considered a little sketchy, but it fixes a real problem and is not too likely to break something important (ha! that's just inviting Murphy in). Please test 2.5.1c1 that will be coming out in a few days. I modified the patch to use a utility function and added comments and docstrings. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685563&group_id=5470 From noreply at sourceforge.net Sun Apr 1 23:18:00 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 14:18:00 -0700 Subject: [Patches] [ python-Patches-1691032 ] Migrate test_minidom.py to unittest Message-ID: Patches item #1691032, was opened at 2007-03-30 00:15 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: Migrate test_minidom.py to unittest Initial Comment: Update Lib/test/test_minidom.py to use the unittest module. Also: I noticed the test_minidom.py contains some code (now long dead) to check for reference cycles. The code is active only when Node.allnodes exists; this hasn't existed for over four years now. After re-implementing that test using gc.DEBUG_SAVEALL, I found that many tests failed because of missing unlink() calls. I added them. After that, one test was still failing. This is because expatbuilder.parseString() and friends do not clean up some cyclic references on error. The patch includes a fix for that (in Lib/xml/dom/expatbuilder.py). An alternative would be to ditch the cycle-checking test entirely. I can submit a patch that does that instead, if desired. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-01 17:18 Message: Logged In: YES user_id=18139 Originator: YES OK, I've attached two new patches. test_minidom.second.patch - Changes "confirm" to "assert_" or "assertEquals", breaks multi-line assertions into many one-line assertions, adds convenience methods "assertSameNode" and "assertNotSameNode", etc. Also replaces the Node.allnodes test with gc-based test and adds a few calls to unlink(). expatbuilder.second.patch - Fixes the test failure discovered by the new gc testing. File Added: test_minidom.second.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-30 02:49 Message: Logged In: YES user_id=1344176 Originator: NO test_minidom already uses unittest (as of patch #1683397, SVN r54603). Could you update your patch to the latest trunk revision? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 From noreply at sourceforge.net Sun Apr 1 23:18:55 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 14:18:55 -0700 Subject: [Patches] [ python-Patches-1691032 ] Migrate test_minidom.py to unittest Message-ID: Patches item #1691032, was opened at 2007-03-30 00:15 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: Migrate test_minidom.py to unittest Initial Comment: Update Lib/test/test_minidom.py to use the unittest module. Also: I noticed the test_minidom.py contains some code (now long dead) to check for reference cycles. The code is active only when Node.allnodes exists; this hasn't existed for over four years now. After re-implementing that test using gc.DEBUG_SAVEALL, I found that many tests failed because of missing unlink() calls. I added them. After that, one test was still failing. This is because expatbuilder.parseString() and friends do not clean up some cyclic references on error. The patch includes a fix for that (in Lib/xml/dom/expatbuilder.py). An alternative would be to ditch the cycle-checking test entirely. I can submit a patch that does that instead, if desired. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-01 17:18 Message: Logged In: YES user_id=18139 Originator: YES File Added: expatbuilder.second.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-01 17:18 Message: Logged In: YES user_id=18139 Originator: YES OK, I've attached two new patches. test_minidom.second.patch - Changes "confirm" to "assert_" or "assertEquals", breaks multi-line assertions into many one-line assertions, adds convenience methods "assertSameNode" and "assertNotSameNode", etc. Also replaces the Node.allnodes test with gc-based test and adds a few calls to unlink(). expatbuilder.second.patch - Fixes the test failure discovered by the new gc testing. File Added: test_minidom.second.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-30 02:49 Message: Logged In: YES user_id=1344176 Originator: NO test_minidom already uses unittest (as of patch #1683397, SVN r54603). Could you update your patch to the latest trunk revision? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 From noreply at sourceforge.net Sun Apr 1 23:47:56 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 14:47:56 -0700 Subject: [Patches] [ python-Patches-1690201 ] Added support for custom readline functions Message-ID: Patches item #1690201, was opened at 2007-03-28 21:52 Message generated for change (Comment added) made by paulhankin You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ben Timby (btimby) Assigned to: Nobody/Anonymous (nobody) Summary: Added support for custom readline functions Initial Comment: This patch allows for binding a key or keyseq to a custom function named custom. A custom handler can be registered and will be called by readline when the registered sequence input. Example: rl.py -- import readline def handler(): print "need some help?" readline.set_custom_hook(handler) readline.parse_and_bind("\"?\": custom") text = raw_input("type ? for help# ") -- ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 21:47 Message: Logged In: YES user_id=1740099 Originator: NO Gnu readline supports custom functions in C - it would be better to adopt the gnu readline custom function interface to python, rather than using the 'custom' handler which smells of a hack. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 23:01 Message: Logged In: YES user_id=932679 Originator: YES new Example: rl.py -- #!/usr/bin/python import readline def say_hello(): print print "this is my help text..." readline.on_new_line() readline.set_custom_hook(say_hello) readline.parse_and_bind("\"?\": custom") while True: data = raw_input("input# ") print "data: ", data -- ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 23:00 Message: Logged In: YES user_id=932679 Originator: YES File Added: python-2.3.4-readline_custom.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 From noreply at sourceforge.net Sun Apr 1 23:50:49 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 14:50:49 -0700 Subject: [Patches] [ python-Patches-1692335 ] Move initial args assignment to BaseException.__new__ Message-ID: Patches item #1692335, was opened at 2007-04-01 15:46 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692335&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: ?iga Seilnacht (zseil) Assigned to: Nobody/Anonymous (nobody) Summary: Move initial args assignment to BaseException.__new__ Initial Comment: Pickling exceptions fails when an Exception class requires an argument in the constructor, but doesn't call its base class' constructor. See this mail for details: http://mail.python.org/pipermail/python-dev/2007-April/072416.html This patch simply moves initial args assignment to BaseException.__new__. This should fix most of the problems, because it is very unlikely that an exception overwrites the __new__ method; exceptions used to be old style classes, which don't support the __new__ special method. The args attribute is still overwritten in all the __init__ methods, so there shouldn't be any backward compatibility problems. ---------------------------------------------------------------------- >Comment By: ?iga Seilnacht (zseil) Date: 2007-04-01 23:50 Message: Logged In: YES user_id=1326842 Originator: YES I'm attaching a test that Eric Huss sent in private mail. The test fails, because old exception pickles don't have args for the reconstructor, but their __init__() gets called anyway because they are new style classes now. The problem is in cPickle.InstanceNew() and pickle.Unpickler._instantiate(). Those methods behave differently depending on the type of the object instantiated; they avoid the __init__() call when type is an old style class. There is nothing that can be done in the exception classes to fix this issue; the fix would need to be in the pickle and cPickle module. File Added: test_exception_pickle.py ---------------------------------------------------------------------- Comment By: ?iga Seilnacht (zseil) Date: 2007-04-01 15:47 Message: Logged In: YES user_id=1326842 Originator: YES File Added: exc_args_25.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692335&group_id=5470 From noreply at sourceforge.net Mon Apr 2 03:53:40 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 18:53:40 -0700 Subject: [Patches] [ python-Patches-1690201 ] Added support for custom readline functions Message-ID: Patches item #1690201, was opened at 2007-03-28 17:52 Message generated for change (Comment added) made by btimby You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ben Timby (btimby) Assigned to: Nobody/Anonymous (nobody) Summary: Added support for custom readline functions Initial Comment: This patch allows for binding a key or keyseq to a custom function named custom. A custom handler can be registered and will be called by readline when the registered sequence input. Example: rl.py -- import readline def handler(): print "need some help?" readline.set_custom_hook(handler) readline.parse_and_bind("\"?\": custom") text = raw_input("type ? for help# ") -- ---------------------------------------------------------------------- >Comment By: Ben Timby (btimby) Date: 2007-04-01 21:53 Message: Logged In: YES user_id=932679 Originator: YES If by adopt you mean adapt, that is what I have done. I simplified things to allow a single named function "custom" but it uses the gnu readline custom function interface (rl_add_defunc) . Have you even looked at the patch? ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 17:47 Message: Logged In: YES user_id=1740099 Originator: NO Gnu readline supports custom functions in C - it would be better to adopt the gnu readline custom function interface to python, rather than using the 'custom' handler which smells of a hack. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 19:01 Message: Logged In: YES user_id=932679 Originator: YES new Example: rl.py -- #!/usr/bin/python import readline def say_hello(): print print "this is my help text..." readline.on_new_line() readline.set_custom_hook(say_hello) readline.parse_and_bind("\"?\": custom") while True: data = raw_input("input# ") print "data: ", data -- ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 19:00 Message: Logged In: YES user_id=932679 Originator: YES File Added: python-2.3.4-readline_custom.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 From noreply at sourceforge.net Mon Apr 2 07:23:18 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 01 Apr 2007 22:23:18 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 23:23 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Mon Apr 2 10:41:15 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 01:41:15 -0700 Subject: [Patches] [ python-Patches-850482 ] Enhance frame handing in warnings.warn() Message-ID: Patches item #850482, was opened at 2003-11-28 00:12 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=850482&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Enhance frame handing in warnings.warn() Initial Comment: This patch enhances warnings.warn() in the following way: The stacklevel passed in may be negative. In that case the call stack is searched for the innermost frame whose module name differs in the first -stacklevel components. This frame will be used in the report. So when you have the following call stack: m1.f() m1.m11.f() m1.m12.f() m2.m21.f() m2.m22.f() and the innermost function() m2.m22.f() call warnings.warn() with a stacklevel of -2 the frame reported will be from m1.m12.f(), because it is the first one from outside the m2 package. ---------------------------------------------------------------------- >Comment By: ?iga Seilnacht (zseil) Date: 2007-04-02 10:41 Message: Logged In: YES user_id=1326842 Originator: NO I don't know how desireable this feature is, but I think that a better solution would be to add a new parameter to warnings.warn(), instead of reusing the old one. Also, your description is wrong; when warn() in your example is called with stacklevel -2, the reported frame is m2.m21.f(). To get your result, you have to call warn() with stacklevel -1. I think that a better solution would be to add a skipmodules='moduleprefix' parameter to the warn() function. warn() would then simply look for the first frame whose module name doesn't start with this prefix. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=850482&group_id=5470 From noreply at sourceforge.net Mon Apr 2 11:54:44 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 02:54:44 -0700 Subject: [Patches] [ python-Patches-1197318 ] Cygwin case-sensitive import patch Message-ID: Patches item #1197318, was opened at 2005-05-07 17:56 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1197318&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Jason Tishler (jlt63) Assigned to: Jason Tishler (jlt63) Summary: Cygwin case-sensitive import patch Initial Comment: A problem regarding importing symlinked modules was recently reported on the Cygwin mailing list: http://cygwin.com/ml/cygwin/2005-04/msg00257.html The following test case demonstrates the problem: $ ls -l total 1 lrwxrwxrwx 1 jt None 6 Apr 23 13:32 bar.py -> foo.py -rw-r--r-- 1 jt None 24 Apr 18 20:13 foo.py $ python -c 'import bar' Traceback (most recent call last): File "", line 1, in ? ImportError: No module named bar Since Cygwin's case_ok() uses a modified version of the Windows's version, the symlinked bar module actually resolves to file foo.py instead of bar.py. This obviously causes the matching code to fail (regardless of case). The attached patch fixes this problem. All that it does it make Cygwin use the Mac OS X case_ok() instead of a modified Window's version. Is this OK to apply? Or, would someone like to make sure I didn't break the Windows and Mac builds first? AFAICT, I didn't, but you never know... :,) Additionally, can I apply this patch to the 2.4 maintence branch? If so, is "release24-maint" the correct CVS tag to use? Thanks. ---------------------------------------------------------------------- >Comment By: ?iga Seilnacht (zseil) Date: 2007-04-02 11:54 Message: Logged In: YES user_id=1326842 Originator: NO The 2.4 branch is not maintained anymore, so this patch can be closed. The patch is included in Python 2.5. ---------------------------------------------------------------------- Comment By: Jason Tishler (jlt63) Date: 2005-05-20 03:06 Message: Logged In: YES user_id=86216 Commited onto HEAD as Python/import.c 2.241, so I'm in forgiveness mode for this change... :,) However, I won't commit onto the maintence branch without explicit permission, so I'm still in permission mode for this one... :,) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1197318&group_id=5470 From noreply at sourceforge.net Mon Apr 2 15:54:04 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 06:54:04 -0700 Subject: [Patches] [ python-Patches-1159139 ] cgi.py invalid REQUEST_METHOD set Message-ID: Patches item #1159139, was opened at 2005-03-08 11:04 Message generated for change (Comment added) made by joesalmeri You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Closed Resolution: Rejected Priority: 5 Private: No Submitted By: Joe (joesalmeri) >Assigned to: Martin v. L?wis (loewis) Summary: cgi.py invalid REQUEST_METHOD set Initial Comment: Python Version 2.4 OS Windows XP SP 2 + WindowsUpdates Patch for lastest CVS Tree When the environment does not have a correctly set REQUEST_METHOD cgi.py prompts for key=value pairs by reading from sys.stdin. After the values are read from sys.stdin they are never stored in the FieldStorage.list attribute like they are when the FieldStorage.read_urlencoded or FieldStorage.read_multi methods are called. This causes a problem when FieldStorage.keys() is called because although the values were read from sys.stdin they were never stored in FieldStorage.list. Although you could argue that REQUEST_METHOD should have been set correctly in the first place, it still seems like if cgi.py is going to handle that situation by actually reading the values from sys.stdin it should store them too. This fix stores the values read. ---------------------------------------------------------------------- >Comment By: Joe (joesalmeri) Date: 2007-04-02 09:54 Message: Logged In: YES user_id=1235002 Originator: YES It been a long time since I submitted this patch but I meant that REQUEST_METHOD is not set although I believe it also happens when it is set incorrectly. Regarding your question with the prompting I thought I explained that in the original post but here are more details: When the REQUEST_METHOD is not set the __init__ method for the FieldStorage class ends up setting the self.fp to sys.stdin. When the read_single method is called it calls self.read_lines(). Since self.fp is equal to sys.stdin at that point and since there is not input pending on stdin it ends up prompting for input until it reaches EOF. The bug was that although it prompted for input by reading from sys.stdin, the values that it read were never stored in FieldStorage like they are when the values are read from the normal sources. Since someone went to the trouble of coding cgi.py to read the values from stdin when the REQUEST_METHOD was invalid it seems to make sense to store them. As jimjjewett point out this addresses the issue of allowing a cgi script to be called from the command line which is useful for debugging of if no web server is running. Since it is counter-intuitive to read the values or not store them the other option would be not to set self.fp = sys.stdin and instead through an error that REQUEST_METHOD was not set properly however, it seems more benefitial to read from sys.stdin and then store the values read in FieldStorage. Does that make more sense now? Thanks. ---------------------------------------------------------------------- Comment By: SourceForge Robot (sf-robot) Date: 2007-03-20 22:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-06 08:23 Message: Logged In: YES user_id=21627 Originator: NO I must be completely missing the point of this patch. Where in the code does it currently prompt for key=value pairs if REQUEST_METHOD is not correctly set? By "not correctly set", do you mean "not set" or "set incorrectly"? If the latter, to what value? You seem to be modifying the read_single function. This is meant to read a single body of the CGI request (as sent by the HTTP client), not the key-value-pairs. I've tried to come up with a demo application of this new functionality, but failed: I couldn't make this new code do anything useful. What Python script should I use, how should I invoke it, what environment variables should I give, and what standard input? Tentatively rejecting the patch. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2005-03-14 17:54 Message: Logged In: YES user_id=764593 +1 on supporting use without a REQUEST_METHOD, if only to match Perl on "what happens if my .cgi was called straight from the command line"? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 From noreply at sourceforge.net Mon Apr 2 15:54:56 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 06:54:56 -0700 Subject: [Patches] [ python-Patches-1159139 ] cgi.py invalid REQUEST_METHOD set Message-ID: Patches item #1159139, was opened at 2005-03-08 11:04 Message generated for change (Settings changed) made by joesalmeri You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Open Resolution: Rejected Priority: 5 Private: No Submitted By: Joe (joesalmeri) Assigned to: Martin v. L?wis (loewis) Summary: cgi.py invalid REQUEST_METHOD set Initial Comment: Python Version 2.4 OS Windows XP SP 2 + WindowsUpdates Patch for lastest CVS Tree When the environment does not have a correctly set REQUEST_METHOD cgi.py prompts for key=value pairs by reading from sys.stdin. After the values are read from sys.stdin they are never stored in the FieldStorage.list attribute like they are when the FieldStorage.read_urlencoded or FieldStorage.read_multi methods are called. This causes a problem when FieldStorage.keys() is called because although the values were read from sys.stdin they were never stored in FieldStorage.list. Although you could argue that REQUEST_METHOD should have been set correctly in the first place, it still seems like if cgi.py is going to handle that situation by actually reading the values from sys.stdin it should store them too. This fix stores the values read. ---------------------------------------------------------------------- Comment By: Joe (joesalmeri) Date: 2007-04-02 09:54 Message: Logged In: YES user_id=1235002 Originator: YES It been a long time since I submitted this patch but I meant that REQUEST_METHOD is not set although I believe it also happens when it is set incorrectly. Regarding your question with the prompting I thought I explained that in the original post but here are more details: When the REQUEST_METHOD is not set the __init__ method for the FieldStorage class ends up setting the self.fp to sys.stdin. When the read_single method is called it calls self.read_lines(). Since self.fp is equal to sys.stdin at that point and since there is not input pending on stdin it ends up prompting for input until it reaches EOF. The bug was that although it prompted for input by reading from sys.stdin, the values that it read were never stored in FieldStorage like they are when the values are read from the normal sources. Since someone went to the trouble of coding cgi.py to read the values from stdin when the REQUEST_METHOD was invalid it seems to make sense to store them. As jimjjewett point out this addresses the issue of allowing a cgi script to be called from the command line which is useful for debugging of if no web server is running. Since it is counter-intuitive to read the values or not store them the other option would be not to set self.fp = sys.stdin and instead through an error that REQUEST_METHOD was not set properly however, it seems more benefitial to read from sys.stdin and then store the values read in FieldStorage. Does that make more sense now? Thanks. ---------------------------------------------------------------------- Comment By: SourceForge Robot (sf-robot) Date: 2007-03-20 22:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-06 08:23 Message: Logged In: YES user_id=21627 Originator: NO I must be completely missing the point of this patch. Where in the code does it currently prompt for key=value pairs if REQUEST_METHOD is not correctly set? By "not correctly set", do you mean "not set" or "set incorrectly"? If the latter, to what value? You seem to be modifying the read_single function. This is meant to read a single body of the CGI request (as sent by the HTTP client), not the key-value-pairs. I've tried to come up with a demo application of this new functionality, but failed: I couldn't make this new code do anything useful. What Python script should I use, how should I invoke it, what environment variables should I give, and what standard input? Tentatively rejecting the patch. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2005-03-14 17:54 Message: Logged In: YES user_id=764593 +1 on supporting use without a REQUEST_METHOD, if only to match Perl on "what happens if my .cgi was called straight from the command line"? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 From noreply at sourceforge.net Mon Apr 2 16:14:22 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 07:14:22 -0700 Subject: [Patches] [ python-Patches-1159139 ] cgi.py invalid REQUEST_METHOD set Message-ID: Patches item #1159139, was opened at 2005-03-08 11:04 Message generated for change (Comment added) made by joesalmeri You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: Rejected Priority: 5 Private: No Submitted By: Joe (joesalmeri) Assigned to: Martin v. L?wis (loewis) Summary: cgi.py invalid REQUEST_METHOD set Initial Comment: Python Version 2.4 OS Windows XP SP 2 + WindowsUpdates Patch for lastest CVS Tree When the environment does not have a correctly set REQUEST_METHOD cgi.py prompts for key=value pairs by reading from sys.stdin. After the values are read from sys.stdin they are never stored in the FieldStorage.list attribute like they are when the FieldStorage.read_urlencoded or FieldStorage.read_multi methods are called. This causes a problem when FieldStorage.keys() is called because although the values were read from sys.stdin they were never stored in FieldStorage.list. Although you could argue that REQUEST_METHOD should have been set correctly in the first place, it still seems like if cgi.py is going to handle that situation by actually reading the values from sys.stdin it should store them too. This fix stores the values read. ---------------------------------------------------------------------- >Comment By: Joe (joesalmeri) Date: 2007-04-02 10:14 Message: Logged In: YES user_id=1235002 Originator: YES Test Script that shows problem: # Start of Script import cgi import os os.environ['REQUEST_METHOD'] = 'cgi' field = cgi.FieldStorage() print field print field.keys() for k in field.keys(): print k, field[k] # End Of Script When you run that script it will hang because it is waiting on the read from self.fp (sys.stdin), press ctrl-z (EOF) on Windows and hit enter (or you could enter some key/value pairs but they are not stored in FieldStorage). cgi.py will then fail as follows: ^Z FieldStorage(None, None, '') Traceback (most recent call last): File "Q:\cgifix\FixTest\t2.py", line 9, in ? print field.keys() File "P:\SW\Python\lib\cgi.py", line 601, in keys raise TypeError, "not indexable" TypeError: not indexable Now modify the above script to use the NEW cgi.py I provided. When prompted for the values on sys.stdin enter your pairs enter the following (press enter after each line) a=1 b=2 c=3 now press ctrl-z (EOF) and enter and you will see that cgi.py has been patch to fix the bug. FieldStorage(None, None, 'a=1\nb=2\nc=3\n') ['a', 'b', 'c'] a MiniFieldStorage('a', '1') b MiniFieldStorage('b', '2') c MiniFieldStorage('c', '3') I hope that makes the problem and solution more clear so that this can be included in the next release. Thanks! ---------------------------------------------------------------------- Comment By: Joe (joesalmeri) Date: 2007-04-02 09:54 Message: Logged In: YES user_id=1235002 Originator: YES It been a long time since I submitted this patch but I meant that REQUEST_METHOD is not set although I believe it also happens when it is set incorrectly. Regarding your question with the prompting I thought I explained that in the original post but here are more details: When the REQUEST_METHOD is not set the __init__ method for the FieldStorage class ends up setting the self.fp to sys.stdin. When the read_single method is called it calls self.read_lines(). Since self.fp is equal to sys.stdin at that point and since there is not input pending on stdin it ends up prompting for input until it reaches EOF. The bug was that although it prompted for input by reading from sys.stdin, the values that it read were never stored in FieldStorage like they are when the values are read from the normal sources. Since someone went to the trouble of coding cgi.py to read the values from stdin when the REQUEST_METHOD was invalid it seems to make sense to store them. As jimjjewett point out this addresses the issue of allowing a cgi script to be called from the command line which is useful for debugging of if no web server is running. Since it is counter-intuitive to read the values or not store them the other option would be not to set self.fp = sys.stdin and instead through an error that REQUEST_METHOD was not set properly however, it seems more benefitial to read from sys.stdin and then store the values read in FieldStorage. Does that make more sense now? Thanks. ---------------------------------------------------------------------- Comment By: SourceForge Robot (sf-robot) Date: 2007-03-20 22:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-06 08:23 Message: Logged In: YES user_id=21627 Originator: NO I must be completely missing the point of this patch. Where in the code does it currently prompt for key=value pairs if REQUEST_METHOD is not correctly set? By "not correctly set", do you mean "not set" or "set incorrectly"? If the latter, to what value? You seem to be modifying the read_single function. This is meant to read a single body of the CGI request (as sent by the HTTP client), not the key-value-pairs. I've tried to come up with a demo application of this new functionality, but failed: I couldn't make this new code do anything useful. What Python script should I use, how should I invoke it, what environment variables should I give, and what standard input? Tentatively rejecting the patch. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2005-03-14 17:54 Message: Logged In: YES user_id=764593 +1 on supporting use without a REQUEST_METHOD, if only to match Perl on "what happens if my .cgi was called straight from the command line"? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 From noreply at sourceforge.net Mon Apr 2 19:28:07 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 10:28:07 -0700 Subject: [Patches] [ python-Patches-1690201 ] Added support for custom readline functions Message-ID: Patches item #1690201, was opened at 2007-03-28 21:52 Message generated for change (Comment added) made by paulhankin You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ben Timby (btimby) Assigned to: Nobody/Anonymous (nobody) Summary: Added support for custom readline functions Initial Comment: This patch allows for binding a key or keyseq to a custom function named custom. A custom handler can be registered and will be called by readline when the registered sequence input. Example: rl.py -- import readline def handler(): print "need some help?" readline.set_custom_hook(handler) readline.parse_and_bind("\"?\": custom") text = raw_input("type ? for help# ") -- ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-02 17:28 Message: Logged In: YES user_id=1740099 Originator: NO Hi Ben, I'm sorry if you thought my short reply suggested I hadn't looked at the patch properly. I'll try to clarify what I meant. Gnu readline provides an interface to custom functions: you declare a function that takes a 'count' and a 'key' and returns 0 or an error. You then call rl_add_defunc to declare it. You use the name you used in 'rl_add_defunc' in the readline init file to bind the command to a key. The patch uses a different interface: the user provides a function that takes no arguments. He declares it by calling 'set_custom_hook', and calls it by using the name 'custom' in the init file to bind it to a key. An interface consistent between python and readline would be: def say_hello(key, count): ... readline.add_defunc("say_hello", say_hello) readline.parse_and_bind('"?" : say_hello') (Perhaps the 'add_defun' is an artefact of lack of dynamic features of C and would be dropped from the python interface). The patch's use of a 'custom' hook is more complicated than using the function name directly in the init file, and provides less functionality (a single command, and removes the arguments that the c command would have). Accepting this patch will cause minor compatibility trouble in future if anyone works out a way round the admittedly difficult technical problems of doing things the 'right' way. Given the interface troubles, I'm not convinced enough of the usefulness of the patch. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-04-02 01:53 Message: Logged In: YES user_id=932679 Originator: YES If by adopt you mean adapt, that is what I have done. I simplified things to allow a single named function "custom" but it uses the gnu readline custom function interface (rl_add_defunc) . Have you even looked at the patch? ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 21:47 Message: Logged In: YES user_id=1740099 Originator: NO Gnu readline supports custom functions in C - it would be better to adopt the gnu readline custom function interface to python, rather than using the 'custom' handler which smells of a hack. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 23:01 Message: Logged In: YES user_id=932679 Originator: YES new Example: rl.py -- #!/usr/bin/python import readline def say_hello(): print print "this is my help text..." readline.on_new_line() readline.set_custom_hook(say_hello) readline.parse_and_bind("\"?\": custom") while True: data = raw_input("input# ") print "data: ", data -- ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 23:00 Message: Logged In: YES user_id=932679 Originator: YES File Added: python-2.3.4-readline_custom.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 From noreply at sourceforge.net Mon Apr 2 20:46:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 11:46:33 -0700 Subject: [Patches] [ python-Patches-1693149 ] trace.py --ignore-module should accept module name list. Message-ID: Patches item #1693149, was opened at 2007-04-02 14:46 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1693149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Nobody/Anonymous (nobody) Summary: trace.py --ignore-module should accept module name list. Initial Comment: Currently, trace.py's command line option "--ignore-module" only accepts one module name. If multiple module names need to be passed, this option can be used multiple times. However, this patch makes this process simple by causing trace.py to accept comma separated module names. Please note that it is still possible to use this option multiple times but each time, a comma separated list can be passed. Usage example: $ python -m trace -t --ignore-module tempfile,posixpath --ignore-module random tfile.py On a separate note, the patch also has slight change to "--ignore-dir"'s description to bring it up to date with what "--help" shows. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1693149&group_id=5470 From noreply at sourceforge.net Mon Apr 2 22:47:15 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 13:47:15 -0700 Subject: [Patches] [ python-Patches-1159139 ] cgi.py invalid REQUEST_METHOD set Message-ID: Patches item #1159139, was opened at 2005-03-08 11:04 Message generated for change (Settings changed) made by joesalmeri You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open >Resolution: None Priority: 5 Private: No Submitted By: Joe (joesalmeri) Assigned to: Martin v. L?wis (loewis) Summary: cgi.py invalid REQUEST_METHOD set Initial Comment: Python Version 2.4 OS Windows XP SP 2 + WindowsUpdates Patch for lastest CVS Tree When the environment does not have a correctly set REQUEST_METHOD cgi.py prompts for key=value pairs by reading from sys.stdin. After the values are read from sys.stdin they are never stored in the FieldStorage.list attribute like they are when the FieldStorage.read_urlencoded or FieldStorage.read_multi methods are called. This causes a problem when FieldStorage.keys() is called because although the values were read from sys.stdin they were never stored in FieldStorage.list. Although you could argue that REQUEST_METHOD should have been set correctly in the first place, it still seems like if cgi.py is going to handle that situation by actually reading the values from sys.stdin it should store them too. This fix stores the values read. ---------------------------------------------------------------------- Comment By: Joe (joesalmeri) Date: 2007-04-02 10:14 Message: Logged In: YES user_id=1235002 Originator: YES Test Script that shows problem: # Start of Script import cgi import os os.environ['REQUEST_METHOD'] = 'cgi' field = cgi.FieldStorage() print field print field.keys() for k in field.keys(): print k, field[k] # End Of Script When you run that script it will hang because it is waiting on the read from self.fp (sys.stdin), press ctrl-z (EOF) on Windows and hit enter (or you could enter some key/value pairs but they are not stored in FieldStorage). cgi.py will then fail as follows: ^Z FieldStorage(None, None, '') Traceback (most recent call last): File "Q:\cgifix\FixTest\t2.py", line 9, in ? print field.keys() File "P:\SW\Python\lib\cgi.py", line 601, in keys raise TypeError, "not indexable" TypeError: not indexable Now modify the above script to use the NEW cgi.py I provided. When prompted for the values on sys.stdin enter your pairs enter the following (press enter after each line) a=1 b=2 c=3 now press ctrl-z (EOF) and enter and you will see that cgi.py has been patch to fix the bug. FieldStorage(None, None, 'a=1\nb=2\nc=3\n') ['a', 'b', 'c'] a MiniFieldStorage('a', '1') b MiniFieldStorage('b', '2') c MiniFieldStorage('c', '3') I hope that makes the problem and solution more clear so that this can be included in the next release. Thanks! ---------------------------------------------------------------------- Comment By: Joe (joesalmeri) Date: 2007-04-02 09:54 Message: Logged In: YES user_id=1235002 Originator: YES It been a long time since I submitted this patch but I meant that REQUEST_METHOD is not set although I believe it also happens when it is set incorrectly. Regarding your question with the prompting I thought I explained that in the original post but here are more details: When the REQUEST_METHOD is not set the __init__ method for the FieldStorage class ends up setting the self.fp to sys.stdin. When the read_single method is called it calls self.read_lines(). Since self.fp is equal to sys.stdin at that point and since there is not input pending on stdin it ends up prompting for input until it reaches EOF. The bug was that although it prompted for input by reading from sys.stdin, the values that it read were never stored in FieldStorage like they are when the values are read from the normal sources. Since someone went to the trouble of coding cgi.py to read the values from stdin when the REQUEST_METHOD was invalid it seems to make sense to store them. As jimjjewett point out this addresses the issue of allowing a cgi script to be called from the command line which is useful for debugging of if no web server is running. Since it is counter-intuitive to read the values or not store them the other option would be not to set self.fp = sys.stdin and instead through an error that REQUEST_METHOD was not set properly however, it seems more benefitial to read from sys.stdin and then store the values read in FieldStorage. Does that make more sense now? Thanks. ---------------------------------------------------------------------- Comment By: SourceForge Robot (sf-robot) Date: 2007-03-20 22:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-06 08:23 Message: Logged In: YES user_id=21627 Originator: NO I must be completely missing the point of this patch. Where in the code does it currently prompt for key=value pairs if REQUEST_METHOD is not correctly set? By "not correctly set", do you mean "not set" or "set incorrectly"? If the latter, to what value? You seem to be modifying the read_single function. This is meant to read a single body of the CGI request (as sent by the HTTP client), not the key-value-pairs. I've tried to come up with a demo application of this new functionality, but failed: I couldn't make this new code do anything useful. What Python script should I use, how should I invoke it, what environment variables should I give, and what standard input? Tentatively rejecting the patch. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2005-03-14 17:54 Message: Logged In: YES user_id=764593 +1 on supporting use without a REQUEST_METHOD, if only to match Perl on "what happens if my .cgi was called straight from the command line"? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1159139&group_id=5470 From noreply at sourceforge.net Tue Apr 3 00:15:48 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 15:15:48 -0700 Subject: [Patches] [ python-Patches-1693258 ] Fix for duplicate "preferences" menu-OS X Message-ID: Patches item #1693258, was opened at 2007-04-02 17:15 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1693258&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Walzer (wordtech) Assigned to: Nobody/Anonymous (nobody) Summary: Fix for duplicate "preferences" menu-OS X Initial Comment: Version 8.4.14 of Tcl/Tk Aqua (for OS X) hard-codes a "Preferences" item in the Apple menu. As a result, IDLE now has two "Preferences" items--the active one that is coded by default, and an inactive item that is hard-coded by Tk. The attached patch to macosxSupport.py checks the installed version of Tk, and if it is 8.4.14, substitutes the hard-coded Tk "preferences" menu and binds IDLE's "preferences" dialog to this menu item. If Tk is version 8.4.13 or less, leave the menu alone. This patch addresses Python bug #1691411 opened at 2007-03-30. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1693258&group_id=5470 From noreply at sourceforge.net Tue Apr 3 03:43:30 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 18:43:30 -0700 Subject: [Patches] [ python-Patches-1681432 ] Add triangular distribution to random Message-ID: Patches item #1681432, was opened at 2007-03-15 08:09 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1681432&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) >Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Wladmir van der Laan (miathan6) Assigned to: Raymond Hettinger (rhettinger) Summary: Add triangular distribution to random Initial Comment: This patch adds the so called Triangular distribution for random values. It is often used in simulations when an upper and lower bound is known, and a most likely value but not the exact distribution. This distribution interpolates these values linearly. http://en.wikipedia.org/wiki/Triangular_distribution ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 20:43 Message: Logged In: YES user_id=80475 Originator: NO Will get this fixe-up and applied to Py2.6. That is due out any time soon, so I'll keep it on my todo list for a bit. ---------------------------------------------------------------------- Comment By: Mark Dickinson (marketdickinson) Date: 2007-03-17 11:54 Message: Logged In: YES user_id=703403 Originator: NO Looks correct to me, and seems to work well with correct parameters. The arguments to the square roots are never negative; this means that we get silent failure (that is, some distribution that isn't the triangular distribution) if the input parameters don't satisfy left <= center <= right. Maybe the inputs should be checked, and a ValueError raised on bad input? If the test "x < (center-left)/(right-left)" is replaced by "x*(right-left) < (center - left)" then the code does the right thing in the limit-case when left == center == right (that is, it gives a delta distribution at center); currently it'll raise a ZeroDivisionError in this case. It's not clear to me which behaviour should be preferred. ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-03-17 11:12 Message: Logged In: YES user_id=1740099 Originator: NO Some minor quibbles: the distributions in random name their parameters after the most common use in math text books. Is n't that 'a, b, c' here rather than 'left, right, center' (note, different order too)? Admittedly your parameter names are clearer. There's not total consistency in random, but it looks like 'triangularvariate' would be the right name for the function - although my knowledge of stats is sketchy, so perhaps there's a distinction I'm not aware of. Indentation and spacing around operators needs fixing also. ---------------------------------------------------------------------- Comment By: Wladmir van der Laan (miathan6) Date: 2007-03-16 03:57 Message: Logged In: YES user_id=49336 Originator: YES Yes, I will add some tests and docs, although usage is quite straightforward. That's the main reason also for adding this distribution, it has three parameters which are intuitive (left, center and right) the distribution is just a linear interpolation (P(left)=0 and P(center)=1 and P(right)=0 ). I am writing an event simulation and I added it because I was missing some kind of assymetric distribution for timings; in an uniform distribution t-x and t+y would be just as likely as t, in a gaussian distribution you can only set sigma but not the upper and lower bound. With all the other distributions in there I thought it might come in handy for other people too. I also believe it has its uses in sound generation/processing. Sorry for providing a patch against 2.3 and not 2.5 or higher, I cooked it up at work and it seems I have this ancient version here. ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-15 21:22 Message: Logged In: YES user_id=1344176 Originator: NO Could you work up some tests (Lib/test/test_random.py) and docs (Doc/lib/librandom.tex) for this? An explanation of why you think this should be included in the standard library would be helpful, too. Also, it seems you made this patch against Python 2.3. While this particular patch still applied (relatively) cleanly to Python 2.6a0 source, Python 2.3 is no longer supported and most patches made against it have a high probability of inapplicability. See http://www.python.org/dev/faq/ for information on how to obtain a read-only checkout of the latest source code. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1681432&group_id=5470 From noreply at sourceforge.net Tue Apr 3 03:44:42 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 18:44:42 -0700 Subject: [Patches] [ python-Patches-1685986 ] Method cache Message-ID: Patches item #1685986, was opened at 2007-03-22 08:47 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Armin Rigo (arigo) Assigned to: Raymond Hettinger (rhettinger) Summary: Method cache Initial Comment: This is a port of PyPy's method cache optimization. It gives a very good speed-up to PyPy, and it is simple enough to be attempted on CPython. The patch is against Python 2.4, which is the version I still use; I've applied the patch on the system-wide Pythons of a couple of machines a few days ago and haven't had any problem since. It gives a consistent 20+% of speed-up on all applications and benchmarks I tried, and up to 36% in extreme OO examples like the "richards" benchmark. The patch needs to be ported to the HEAD, and it needs some extra careful review. I'm a bit too busy right now to do any of these but I'll eventually come back to it if nobody else does. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 20:44 Message: Logged In: YES user_id=80475 Originator: NO Yes, I've got it from here. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-22 10:28 Message: Logged In: YES user_id=849994 Originator: NO Perhaps Raymond, as one of the optimization specialists, can have a look in the meantime? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 From noreply at sourceforge.net Tue Apr 3 03:52:08 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 18:52:08 -0700 Subject: [Patches] [ python-Patches-1244861 ] Enable os.startfile and webbrowser.WindowsDefault on Cygwin Message-ID: Patches item #1244861, was opened at 2005-07-25 18:38 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1244861&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Hoffman (hoffmanm) >Assigned to: Nobody/Anonymous (nobody) Summary: Enable os.startfile and webbrowser.WindowsDefault on Cygwin Initial Comment: This patch enables os.startfile() on Cygwin, converting Cygwin POSIX paths to Windows paths unless they look like URLs, and calling ShellExecute() on the result. It also enables the webbrowser.WindowsDefault class, which uses os.startfile(). It has to make WindowsError and the PyErr*Windows* functions available as well because win32_startfile() might call them. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 20:52 Message: Logged In: YES user_id=80475 Originator: NO Unfortunately, I no longer have access to a Windows build environment and cannot exercise this patch. ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2006-01-17 10:17 Message: Logged In: YES user_id=987664 Raymond, did you ever have a chance to look at this patch? ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-09-15 08:04 Message: Logged In: YES user_id=1188172 Raymond, can you look at this one? It seems to be okay, but I can't test it. ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-09-15 07:44 Message: Logged In: YES user_id=987664 Let's try this again... ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-09-15 07:39 Message: Logged In: YES user_id=1188172 I still see no patch... are you sure you check the right box? ;-) ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-09-15 05:49 Message: Logged In: YES user_id=987664 Argh, sorry about that. Thanks Reinhold; here's the patch. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-09-15 02:20 Message: Logged In: YES user_id=1188172 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1244861&group_id=5470 From noreply at sourceforge.net Tue Apr 3 03:59:51 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 18:59:51 -0700 Subject: [Patches] [ python-Patches-1492828 ] Improvements to ceval.c Message-ID: Patches item #1492828, was opened at 2006-05-22 05:15 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1492828&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.5 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: mrjbq7 (mrjbq7) Assigned to: Raymond Hettinger (rhettinger) Summary: Improvements to ceval.c Initial Comment: >From Raymond Hettinger, submitting here to keep track of for NeedForSpeed sprint. Here are some customizations to your Python build: First, make sure that WITH_TSC and WITH_THREAD are not defined in the build. Then, attached diff to disable the tracing code, remove NOPs, speed-up absolute jumps, and increase the signal check interval. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 20:59 Message: Logged In: YES user_id=80475 Originator: NO Sorry, all of these are useless for the general distribution. * Increasing the ticker rollovers will give a tiny speed-up but would hurt the responsiveness of multi-threaded applications. * The tracing functions are needed in the general distribution and should not be shut-off. * The NOP case is a fail-safe; not essential, but not detrimental either. The peephole optimizer introduces NOPS and then clears them out. If someone makes a change leaving them in or if they are generating their own bytecode, we don't want the eval-loop to fail. * The JUMP_ABSOLUTE step needs to make a full trip through the eval-loop or else it won't be possible to break out of a "while 1: pass". While ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2006-05-22 10:39 Message: Logged In: YES user_id=31435 Assigned to Raymond. Raymond is there something of general use here? As a standalone patch, it sucks ;-) ---------------------------------------------------------------------- Comment By: mrjbq7 (mrjbq7) Date: 2006-05-22 06:00 Message: Logged In: YES user_id=1172546 Okay, now I checked the box "upload and attach file". Thats a terrible UI. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1492828&group_id=5470 From noreply at sourceforge.net Tue Apr 3 04:17:56 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 19:17:56 -0700 Subject: [Patches] [ python-Patches-1244861 ] Enable os.startfile and webbrowser.WindowsDefault on Cygwin Message-ID: Patches item #1244861, was opened at 2005-07-25 23:38 Message generated for change (Comment added) made by hoffmanm You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1244861&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Hoffman (hoffmanm) Assigned to: Nobody/Anonymous (nobody) Summary: Enable os.startfile and webbrowser.WindowsDefault on Cygwin Initial Comment: This patch enables os.startfile() on Cygwin, converting Cygwin POSIX paths to Windows paths unless they look like URLs, and calling ShellExecute() on the result. It also enables the webbrowser.WindowsDefault class, which uses os.startfile(). It has to make WindowsError and the PyErr*Windows* functions available as well because win32_startfile() might call them. ---------------------------------------------------------------------- >Comment By: Michael Hoffman (hoffmanm) Date: 2007-04-03 02:17 Message: Logged In: YES user_id=987664 Originator: YES Hmmmm. Can you suggest a way to move forward on this? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-03 01:52 Message: Logged In: YES user_id=80475 Originator: NO Unfortunately, I no longer have access to a Windows build environment and cannot exercise this patch. ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2006-01-17 15:17 Message: Logged In: YES user_id=987664 Raymond, did you ever have a chance to look at this patch? ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-09-15 13:04 Message: Logged In: YES user_id=1188172 Raymond, can you look at this one? It seems to be okay, but I can't test it. ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-09-15 12:44 Message: Logged In: YES user_id=987664 Let's try this again... ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-09-15 12:39 Message: Logged In: YES user_id=1188172 I still see no patch... are you sure you check the right box? ;-) ---------------------------------------------------------------------- Comment By: Michael Hoffman (hoffmanm) Date: 2005-09-15 10:49 Message: Logged In: YES user_id=987664 Argh, sorry about that. Thanks Reinhold; here's the patch. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-09-15 07:20 Message: Logged In: YES user_id=1188172 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1244861&group_id=5470 From noreply at sourceforge.net Tue Apr 3 04:33:47 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 19:33:47 -0700 Subject: [Patches] [ python-Patches-1498363 ] Improve super() objects support for implicit method calls Message-ID: Patches item #1498363, was opened at 2006-05-31 12:31 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1498363&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Collin Winter (collinwinter) >Assigned to: Collin Winter (collinwinter) Summary: Improve super() objects support for implicit method calls Initial Comment: The attached patch lets super() objects pass on implicit __getitem__, __setitem__, __delitem__, __len__ and __hash__ calls. For example, to use len() with super() objects, one must currently do something like super(X, X()).__len__() Likewise for __getitem__, super(X, X()).__getitem__(item) That's ugly. This patch lets these be spelled as len(super(X, X())) and super(X, X())[item], respectively. The patch also includes documentation updates and tests for the new functionality. The patch was taken against r46582. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 21:33 Message: Logged In: YES user_id=80475 Originator: NO At one time, I had explored adding some of these methods and then dropped the idea because it left super() with an odd mish-mash of methods that are 1) forwarded 2) not-supported 3) handled directly by the super-object itself. For instance, the repr() of a super-object and is handled directly by the super object. Likewise, the members are part of the super-object and not forwarded. The current state of affairs can be explained (approximately) with the notion that named methods are forwarded but not any of the syntax-assisted implicit calls. This patch clutters that state-of-affairs for a questionable benefit. If you want to push for this, it should be discussed on python-dev so we can reach a concensus on what super-objects should be expected to do and not do. Is it clear that so.__self__ and repr(so) act on the super object while a call like so[x] will traverse the mro? Also, if something like this does go in, please optimize the calls to PyString_FromString() to be invoked no more than once per Python session (otherwise, syntax driven implicit calls may end-up being slower than named method access). ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2006-07-17 08:13 Message: Logged In: YES user_id=1344176 I only added these particular methods because they were a) the only ones I needed at the time, b) the only ones I really had time to implement. I've now got a few more tuits freed up, so I'd be happy to implement more of these methods. As for "explicit is better than implicit", I don't see how that applies here. The same case could be made for all of Python's syntax-assisted method calls. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-07-17 08:05 Message: Logged In: YES user_id=849994 Why only these methods? Why not __add__, __call__ etc.? Implementing all of these methods is a pain and adds considerable complexity to typeobject.c. Frankly, I don't see an improvement since you normally only call super.__getitem__ in __getitem__, and explicit is better than implicit. Raymond, do you have a second opinion? ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2006-06-27 07:22 Message: Logged In: YES user_id=1344176 The patch has been updated to reflect the current SVN state, r47124. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1498363&group_id=5470 From noreply at sourceforge.net Tue Apr 3 04:50:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 19:50:12 -0700 Subject: [Patches] [ python-Patches-1569291 ] Speed-up in array_repeat() Message-ID: Patches item #1569291, was opened at 2006-10-02 08:30 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1569291&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Lars Skovlund (lskovlund) Assigned to: Raymond Hettinger (rhettinger) Summary: Speed-up in array_repeat() Initial Comment: Use iterative doubling when extending the old array. This results in O(log n) calls to memcpy() instead of O(n). ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 21:50 Message: Logged In: YES user_id=80475 Originator: NO This proposal is basically fine. The patch should be re-worked to model as closely as possible the code for string_repeat in Objects/stringobject.c (revisions 30616 and 30368 provide the details). That code is known to work and to have provided a meaningful speed-up. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2006-10-13 10:26 Message: Logged In: YES user_id=341410 This patch has nothing to do with array resizing. It has to do with array initialization. ---------------------------------------------------------------------- Comment By: Larry Hastings (lhastings) Date: 2006-10-13 03:17 Message: Logged In: YES user_id=364875 I'd assumed Python *didn't* double the size when resizing an array because of how much memory that wastes. May I suggest trying it with a multiplier of 1.5x, and comparing both CPU time and memory consumption? I find for these things that 1.5x is nearly as fast and wastes far less memory. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-10-07 19:59 Message: Logged In: YES user_id=80475 This patch is basically fine. Will neaten it up to match our source coding conventions and apply it when I get a chance. Py2.6 won't be out for a good while, so there is no hurry. ---------------------------------------------------------------------- Comment By: Lars Skovlund (lskovlund) Date: 2006-10-07 17:12 Message: Logged In: YES user_id=263372 I wrote this code for a university project I'm doing myself, which involves initializing a *very* large array (it's a remote memory system). It does help there, certainly; for medium-sized arrays, the improvement would be negligable, and for small ones it might even be worse. If you mean, have I benchmarked it with other people's code, no. I just thought I'd offer it to the community, since it has certainly helped me. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2006-10-07 11:39 Message: Logged In: YES user_id=341410 Have you benchmarked this for repeats found "in the wild" to establish *observable* speedup for code that already exists? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1569291&group_id=5470 From noreply at sourceforge.net Tue Apr 3 04:55:44 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 19:55:44 -0700 Subject: [Patches] [ python-Patches-762920 ] API Functions for PyArray Message-ID: Patches item #762920, was opened at 2003-06-29 18:48 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=762920&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) >Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ralph (kohanman) Assigned to: Raymond Hettinger (rhettinger) Summary: API Functions for PyArray Initial Comment: I'd rather send and recieve PyArrays into my C modules since I often end up with a foo(int count, type *data) call anyway. Sure, there's NumPy for this sort of dirtywork, but this is so much more convienient. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 21:55 Message: Logged In: YES user_id=80475 Originator: NO This proposal is basically fine. It will need docs and it can go into Py2.6. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=762920&group_id=5470 From noreply at sourceforge.net Tue Apr 3 05:17:44 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 20:17:44 -0700 Subject: [Patches] [ python-Patches-1054967 ] bdist_deb - Debian packager Message-ID: Patches item #1054967, was opened at 2004-10-27 00:48 Message generated for change (Comment added) made by astraw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1054967&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils and setup.py Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Geoffrey T. Dairiki (dairiki) Assigned to: Sean Reifschneider (jafo) Summary: bdist_deb - Debian packager Initial Comment: Here's a first crack at a bdist_deb. This patch implements two new distutils commands: bdist_deb: Build Debian packages debianize: Create and populate a top-level debian subdirectory. (Essentially dh_make for distutils packages.) There is a slightly detailed README.bdist_deb included in the patches. I'm open to suggestions for improvements and bug-fixes. ---------------------------------------------------------------------- Comment By: Andrew Straw (astraw) Date: 2007-04-03 03:17 Message: Logged In: YES user_id=210276 Originator: NO You may also be interested in stdeb, which produces Debian source packages from Python packages. See http://stdeb.python-hosting.com/ ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-03-28 15:06 Message: Logged In: YES user_id=21627 I agree with most of kbk's comments, except for 5: all of the *_util modules are in distutils, why not deb_util? It is not a command. Also, dairiki, are you willing to sign the contributor form? see http://www.python.org/psf/contrib.html ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-03-19 23:18 Message: Logged In: YES user_id=149084 In response to jafo's request for comments on python-dev: 1. This facility should be documented in the Python Module docs. A section should be introduced in "Distributing Python Modules" after the 5.2 Creating RPM Packages. Also, the reference disutils.command.bdist_deb should be supplied. The fact that the rest of distutils is 'lightly' documented is no excuse :-) 2. The README.bdist_deb should be folded into the doc strings for the two modules (except the change info should be left behind in the patch). 3. The existing doc strings in the two modules are not always in standard form, and could be more extensive. 4. There are triple quoted block comments in the code. These should be # comment blocks, cf. PEP8 and PEP 257. You don't use doc strings to document variables. 5. deb_util.py probably belongs in distutils/command/ 6. I would like it better if the default was to build the source package, and that would be consistent with bdist_rpm's action. 7. I'm not entirely happy with the overall design: it seems there are two ways to go. First, run bdist_deb, which give you a package which isn't debian compliant; there's no debian/changelog. Second, you can run dh_make, which will 'debianize' the Python source. This can only be done once, and then the recommendation is to use the standard Debian tools to create the packages. In this second case, apparently you can't use bdist_deb, nor do you have a convenient way of updating when the Python code changes, if I understand the README.bdist_deb correctly. If these impressions are correct, it's a lot of code for relatively little effect, at least at its current level of development. 8. What about alberanid's last comment? 8. Do we have a commitment from Dairiki to maintain this code for at least a couple of years? ---------------------------------------------------------------------- Comment By: Davide Alberani (alberanid) Date: 2005-02-23 11:10 Message: Logged In: YES user_id=170840 Sometimes you need to install the same package for many different releases of Python; unfortunately this will fail, because dpkg refuses to override the dummy man page, present in both packages. Would it be a good idea to modify line 453 of dh_make to include pyver in the name of the manpage file? ---------------------------------------------------------------------- Comment By: Marek Kubica (leonidasxiv) Date: 2005-01-11 16:53 Message: Logged In: YES user_id=872713 I was looking a long time for something like this. I really hope seeing this in Python 2.5, although I'm not familiar with Python's development process. ---------------------------------------------------------------------- Comment By: Geoffrey T. Dairiki (dairiki) Date: 2004-11-11 16:18 Message: Logged In: YES user_id=45814 Another updated patch set. The only change is a bug fix: when using a hand-build set of debian/* files, dh_make will now deduce the correct Debian revision and build architecture. (Bug reported by Bastian Kleineidam.) Patches, as always are on today's CVS version of distutils. ---------------------------------------------------------------------- Comment By: Davide Alberani (alberanid) Date: 2004-11-09 15:39 Message: Logged In: YES user_id=170840 I've created a patch to make your bdist_deb/dh_make commands compatible with Python 1.5.2; it can be found here: http://erlug.linux.it/~da/tmp/distutils-cvs_debpy152-20041109.patch Beware that I've tested it for a very short time, so double check everything (especially the permissions on file/umasks). Moreover, the patch changes also some other files like build_py.py, install_lib.py, bdist_rpm.py, that were not 1.5.2 compatible - these changes are part of another patch I've posted today. HTH. ---------------------------------------------------------------------- Comment By: Geoffrey T. Dairiki (dairiki) Date: 2004-11-08 21:38 Message: Logged In: YES user_id=45814 Another updated patch set: As suggested by Bastian Kleineidam, when the source distribution includes it's own debian subdirectory, bdist_deb (rather than just crapping out) will now skip the dh_make stage, and just run debuild. As always, patches on the current CVS version of distutils are attached below. ---------------------------------------------------------------------- Comment By: Geoffrey T. Dairiki (dairiki) Date: 2004-11-02 22:20 Message: Logged In: YES user_id=45814 Here's try number three! Changes are mostly fixes for woody. The README.bdist_deb (included in the patch set) enumerates the changes in more detail. (I've bitten the bullet and installed woody on one of my machines, so I'm fairly confident it should work now.) The patches, on todays CVS are, as always, attached below. ==== Note to Davide, Thank you, again, very much for all the testing. A couple of the lintian warnings ('prerm-does-not-remove-usr-doc-link' and 'postinst-does-not-set-usr-doc-link') that are listed in your test suite output are due, I think, to your mixed system. Specifically, I suspect you've got the woody version of lintian (which wants to see symlinks from /usr/doc/ to /usr/share/doc/), but a later version of debhelper. (Woody's dh_installdocs (in the debhelper package) automatically generates those links, later versions don't; /usr/doc has been phased out.) Anyhow, I'm not going to fix those warnings (unless it turns out that you're seeing them on your pure woody system.) On package with python scripts (in /usr/bin) Woody's lintian generates an 'unusual-interpreter' warning which I think is spurious (i.e. it is not a valid warning.) (Lintian 1.20.17 doesn't recognize things like /usr/bin/python2.1 as a valid interpreter --- it seems valid enough to me.) Best Regards, Jeff ---------------------------------------------------------------------- Comment By: Davide Alberani (alberanid) Date: 2004-10-30 08:57 Message: Logged In: YES user_id=170840 I've tried the 29/10 patch with my home system (dpkg-dev 1.10, debhelper 4.1.90 and dh-make 0.30) and another "pure woody" system (dpkg-dev 1.9.21, debhelper 4.0.2 and dh-make 0.30). I've used your test suite and some of my personal projects (using both bdist_deb and dh_make); as far as I can tell, it works very well. :-) bdist_deb always worked as expected, producing good debian packages. I've used dh_make and then I've run "dpkg-buildpackage -rfakeroot" and I got some errors (but I'm not sure they depends on your code). Also your test suite produced some failures. Here you can find the output of "dpkg-buildpackage -rfakeroot" and of your tests suite (both were running on my home system): http://erlug.linux.it/~da/tmp/dpkg-buildpackage http://erlug.linux.it/~da/tmp/testsuite Thank you for your great effort! ---------------------------------------------------------------------- Comment By: Geoffrey T. Dairiki (dairiki) Date: 2004-10-29 19:38 Message: Logged In: YES user_id=45814 Thanks for the comments. Here's a second attempt. Changes include: It might work with woody. (I'd appreciate it if you could try again, Davide) 'debianize' command renamed to 'dh_make'. Use debchange to create debian/changelog. This eliminates the need to duplicate debchange's logic to deduce the packagers name and e-mail. A more complete test script. Patches are on today's CVS. ---------------------------------------------------------------------- Comment By: Davide Alberani (alberanid) Date: 2004-10-27 14:47 Message: Logged In: YES user_id=170840 I've a woody with some packages backported from sarge (debhelper 4.1.90 and dpkg-dev 1.10). Running python2.3 ./setup.py bdist_deb with some of my projects, I got the error: dpkg-buildpackage: unknown option or argument --check-dirname-level=1 Debian dpkg-buildpackage . Commenting out the "check-dirname-level" and "check-dirname-regex" options in the bdist_deb.py file the script can go on, but it exits with the error: debian/rules:11: *** first argument to `word' function must be greater than 0. Stop. Hope this helps. ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2004-10-27 03:44 Message: Logged In: YES user_id=81797 I'm just doing a review of this code. A couple of things: There's been some concern expressed about get_default_maintainer. Namely, that if debchange changes it's algorithm, it won't be reflected in this code. It seems like one possible way around that would be to build a directory with a "debian" directory under it, a fake "changelog", and then call debchange to write the data out, and parse it. Too bad there's not a direct hook into debchange to get that information. Can _formatdate, if email doesn't exist, use rfc822.formatdate()? Ditto for _parseaddr? It looks pretty good. However, when trying to build a .deb of my jotweb2 package, it's failing with: [...] dh_testdir dh_testroot dh_installchangelogs- dh_installdocs cp: cannot stat `doc': No such file or directory dh_installdocs: command returned error code 256 [...] I'm not sure exactly why. I do have a "doc" directory in my main package directory, but I don't reference to it in my setup.py or MANIFEST. Adding it to the MANIFEST doesn't seem to help this. Sean ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1054967&group_id=5470 From noreply at sourceforge.net Tue Apr 3 08:27:01 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 02 Apr 2007 23:27:01 -0700 Subject: [Patches] [ python-Patches-1273829 ] extending os.walk to support following symlinks Message-ID: Patches item #1273829, was opened at 2005-08-26 09:21 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: Accepted Priority: 5 Private: No Submitted By: Erick Tryzelaar (idadesub) Assigned to: Nobody/Anonymous (nobody) Summary: extending os.walk to support following symlinks Initial Comment: this is a very simple extension to os.walk to optionally walk down into symlink directories, when it is supported by the operating system. This patch keeps a compatible interface by having this option disabled. ---------------------------------------------------------------------- >Comment By: ?iga Seilnacht (zseil) Date: 2007-04-03 08:27 Message: Logged In: YES user_id=1326842 Originator: NO Here is a patch that fixes test_os.WalkTests on Windows. I also added a tearDown method to WalkTests and removed Collin's changes to MakedirTests, because they are not needed anymore. File Added: test_os.diff ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-23 16:31 Message: Logged In: YES user_id=21627 Originator: NO The test suite patch (54482) broke the windows build. The assertion self.assertEqual(len(all), 4) fails because it finds 5 directories (link is a directory), then the rest of the code is not executed. In turn, the tear-down code is not run, leaving the directories behind. This causes subsequent tests to fail. When fixing the code, I recommend to move the tear-down code into the tearDown method of the test, to make sure it is always run. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-16 09:22 Message: Logged In: YES user_id=849994 Originator: NO Added docs and tests and committed in rev. 54407. ---------------------------------------------------------------------- Comment By: Stephen Hansen (aptshansen) Date: 2007-03-16 08:34 Message: Logged In: YES user_id=1720594 Originator: NO I'm not sure if this is a good idea; although I am certain it would be useful at times. Dangerous! But if its documented, ... and there's our problem :) Patch applies cleanly to the HEAD, but needs documentation. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-08-26 10:23 Message: Logged In: YES user_id=1188172 The docs to os.walk say: """ Note: On systems that support symbolic links, links to subdirectories appear in dirnames lists, but walk() will not visit them (infinite loops are hard to avoid when following symbolic links). To visit linked directories, you can identify them with os.path.islink(path), and invoke walk(path) on each directly. """ Nevertheless, we could offer this option with a caution message. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 From noreply at sourceforge.net Tue Apr 3 09:00:57 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 03 Apr 2007 00:00:57 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 23:23 Message generated for change (Comment added) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 01:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Tue Apr 3 13:59:10 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 03 Apr 2007 04:59:10 -0700 Subject: [Patches] [ python-Patches-1690201 ] Added support for custom readline functions Message-ID: Patches item #1690201, was opened at 2007-03-28 17:52 Message generated for change (Comment added) made by btimby You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ben Timby (btimby) Assigned to: Nobody/Anonymous (nobody) Summary: Added support for custom readline functions Initial Comment: This patch allows for binding a key or keyseq to a custom function named custom. A custom handler can be registered and will be called by readline when the registered sequence input. Example: rl.py -- import readline def handler(): print "need some help?" readline.set_custom_hook(handler) readline.parse_and_bind("\"?\": custom") text = raw_input("type ? for help# ") -- ---------------------------------------------------------------------- >Comment By: Ben Timby (btimby) Date: 2007-04-03 07:59 Message: Logged In: YES user_id=932679 Originator: YES Paul, thank you for clarifying. I agree with you completely. The reason I was unable to implement the API fully is the following problem (probably as you said due to the lack of dynamic features in C). 1. User defines python function. 2. User calls python readline.add_defun() 3. readline.add_defun() must whip up a C handler that will call the python function and in turn call rl_add_defun to register it. Problem being that multiple custom python functions registered with readline to a single C handler will have no way to determine which python function to call on invocation. 4. User calls readline.parse_and_bind() I have no clue how to do step 3, so I avoided it. If anyone has suggestions, ideas or examples, I am happy to do the work, I simply have not had to do something like this before in C. I don't even know what this would be called in order to google for it. The patch is certainly not as useful as the full API, however, I DID implement it to solve a problem, thus there is some use to it :-). I learned recently that I can maintain this feature as an extension (outside of) python, thus allowing me to continue to use it without depending on python maintainers to perceive usefulness. I will do this for the time being and if I am able to figure out item 3 I will resubmit the patch with a full implementation. ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-02 13:28 Message: Logged In: YES user_id=1740099 Originator: NO Hi Ben, I'm sorry if you thought my short reply suggested I hadn't looked at the patch properly. I'll try to clarify what I meant. Gnu readline provides an interface to custom functions: you declare a function that takes a 'count' and a 'key' and returns 0 or an error. You then call rl_add_defunc to declare it. You use the name you used in 'rl_add_defunc' in the readline init file to bind the command to a key. The patch uses a different interface: the user provides a function that takes no arguments. He declares it by calling 'set_custom_hook', and calls it by using the name 'custom' in the init file to bind it to a key. An interface consistent between python and readline would be: def say_hello(key, count): ... readline.add_defunc("say_hello", say_hello) readline.parse_and_bind('"?" : say_hello') (Perhaps the 'add_defun' is an artefact of lack of dynamic features of C and would be dropped from the python interface). The patch's use of a 'custom' hook is more complicated than using the function name directly in the init file, and provides less functionality (a single command, and removes the arguments that the c command would have). Accepting this patch will cause minor compatibility trouble in future if anyone works out a way round the admittedly difficult technical problems of doing things the 'right' way. Given the interface troubles, I'm not convinced enough of the usefulness of the patch. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-04-01 21:53 Message: Logged In: YES user_id=932679 Originator: YES If by adopt you mean adapt, that is what I have done. I simplified things to allow a single named function "custom" but it uses the gnu readline custom function interface (rl_add_defunc) . Have you even looked at the patch? ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 17:47 Message: Logged In: YES user_id=1740099 Originator: NO Gnu readline supports custom functions in C - it would be better to adopt the gnu readline custom function interface to python, rather than using the 'custom' handler which smells of a hack. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 19:01 Message: Logged In: YES user_id=932679 Originator: YES new Example: rl.py -- #!/usr/bin/python import readline def say_hello(): print print "this is my help text..." readline.on_new_line() readline.set_custom_hook(say_hello) readline.parse_and_bind("\"?\": custom") while True: data = raw_input("input# ") print "data: ", data -- ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 19:00 Message: Logged In: YES user_id=932679 Originator: YES File Added: python-2.3.4-readline_custom.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 From noreply at sourceforge.net Wed Apr 4 07:26:36 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 03 Apr 2007 22:26:36 -0700 Subject: [Patches] [ python-Patches-1691070 ] Fix for bug #1283289 Message-ID: Patches item #1691070, was opened at 2007-03-30 01:45 Message generated for change (Comment added) made by rupole You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Roger Upole (rupole) Assigned to: Nobody/Anonymous (nobody) Summary: Fix for bug #1283289 Initial Comment: This is a fix for [ 1283289 ] PyArg_ParseTupleAndKeywords gives misleading error message. It also yields a 10-15% decrease in cpu usage, depending on the number of arguments. ---------------------------------------------------------------------- >Comment By: Roger Upole (rupole) Date: 2007-04-04 00:26 Message: Logged In: YES user_id=771074 Originator: YES File Added: getargs_1.patch ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 13:00 Message: Logged In: YES user_id=1740099 Originator: NO Patched code compiles and passes test suite, and looks cleaner than the code it replaces. It works on the cases the bug report. There should be unit tests added to Lib/test/test_getargs2; I've skimmed the tests in there and can't see any that test combinations of kw, tuple and positional arguments. The patch replaces significant chunks of arg parsing code, so I'd be more convinced of its correctness if there were some unit tests. Certainly there should be at least one test for the bug the patch fixes. The code is 99% written in the prevailing style, but: Source contains several over-long lines, and non-standard comments, ie you should use /* XXX something .. */ and not /* ??? something .. ??? */ Also there are a few minor spacing inconsistencies which would be nice to tidy up. The diff is one level too high up - it contains the directory Python2.5/ It applies cleanly to the trunk though, it just makes it slightly more difficult to apply the patch. The original bug report requests a post to python-dev describing the new error messages. I suggest this post includes all cases where the new code produces a different message than the old. I suggest to proceed: 1. Add unit tests, both for the bug-fix and for existing functionality which has been modified 2. Fix up comments and long lines 3. Check with python-dev about new error messages ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 From noreply at sourceforge.net Wed Apr 4 07:39:25 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 03 Apr 2007 22:39:25 -0700 Subject: [Patches] [ python-Patches-1691070 ] Fix for bug #1283289 Message-ID: Patches item #1691070, was opened at 2007-03-30 01:45 Message generated for change (Comment added) made by rupole You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Roger Upole (rupole) Assigned to: Nobody/Anonymous (nobody) Summary: Fix for bug #1283289 Initial Comment: This is a fix for [ 1283289 ] PyArg_ParseTupleAndKeywords gives misleading error message. It also yields a 10-15% decrease in cpu usage, depending on the number of arguments. ---------------------------------------------------------------------- >Comment By: Roger Upole (rupole) Date: 2007-04-04 00:39 Message: Logged In: YES user_id=771074 Originator: YES File Added: _testcapimodule.patch ---------------------------------------------------------------------- Comment By: Roger Upole (rupole) Date: 2007-04-04 00:26 Message: Logged In: YES user_id=771074 Originator: YES File Added: getargs_1.patch ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 13:00 Message: Logged In: YES user_id=1740099 Originator: NO Patched code compiles and passes test suite, and looks cleaner than the code it replaces. It works on the cases the bug report. There should be unit tests added to Lib/test/test_getargs2; I've skimmed the tests in there and can't see any that test combinations of kw, tuple and positional arguments. The patch replaces significant chunks of arg parsing code, so I'd be more convinced of its correctness if there were some unit tests. Certainly there should be at least one test for the bug the patch fixes. The code is 99% written in the prevailing style, but: Source contains several over-long lines, and non-standard comments, ie you should use /* XXX something .. */ and not /* ??? something .. ??? */ Also there are a few minor spacing inconsistencies which would be nice to tidy up. The diff is one level too high up - it contains the directory Python2.5/ It applies cleanly to the trunk though, it just makes it slightly more difficult to apply the patch. The original bug report requests a post to python-dev describing the new error messages. I suggest this post includes all cases where the new code produces a different message than the old. I suggest to proceed: 1. Add unit tests, both for the bug-fix and for existing functionality which has been modified 2. Fix up comments and long lines 3. Check with python-dev about new error messages ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 From noreply at sourceforge.net Wed Apr 4 11:40:49 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 02:40:49 -0700 Subject: [Patches] [ python-Patches-1691070 ] Fix for bug #1283289 Message-ID: Patches item #1691070, was opened at 2007-03-30 01:45 Message generated for change (Comment added) made by rupole You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Roger Upole (rupole) Assigned to: Nobody/Anonymous (nobody) Summary: Fix for bug #1283289 Initial Comment: This is a fix for [ 1283289 ] PyArg_ParseTupleAndKeywords gives misleading error message. It also yields a 10-15% decrease in cpu usage, depending on the number of arguments. ---------------------------------------------------------------------- >Comment By: Roger Upole (rupole) Date: 2007-04-04 04:40 Message: Logged In: YES user_id=771074 Originator: YES File Added: test_getargs2.patch ---------------------------------------------------------------------- Comment By: Roger Upole (rupole) Date: 2007-04-04 00:39 Message: Logged In: YES user_id=771074 Originator: YES File Added: _testcapimodule.patch ---------------------------------------------------------------------- Comment By: Roger Upole (rupole) Date: 2007-04-04 00:26 Message: Logged In: YES user_id=771074 Originator: YES File Added: getargs_1.patch ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 13:00 Message: Logged In: YES user_id=1740099 Originator: NO Patched code compiles and passes test suite, and looks cleaner than the code it replaces. It works on the cases the bug report. There should be unit tests added to Lib/test/test_getargs2; I've skimmed the tests in there and can't see any that test combinations of kw, tuple and positional arguments. The patch replaces significant chunks of arg parsing code, so I'd be more convinced of its correctness if there were some unit tests. Certainly there should be at least one test for the bug the patch fixes. The code is 99% written in the prevailing style, but: Source contains several over-long lines, and non-standard comments, ie you should use /* XXX something .. */ and not /* ??? something .. ??? */ Also there are a few minor spacing inconsistencies which would be nice to tidy up. The diff is one level too high up - it contains the directory Python2.5/ It applies cleanly to the trunk though, it just makes it slightly more difficult to apply the patch. The original bug report requests a post to python-dev describing the new error messages. I suggest this post includes all cases where the new code produces a different message than the old. I suggest to proceed: 1. Add unit tests, both for the bug-fix and for existing functionality which has been modified 2. Fix up comments and long lines 3. Check with python-dev about new error messages ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 From noreply at sourceforge.net Wed Apr 4 12:31:16 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 03:31:16 -0700 Subject: [Patches] [ python-Patches-1521196 ] smtplib login fails with aol smtp server Message-ID: Patches item #1521196, was opened at 2006-07-12 09:32 Message generated for change (Comment added) made by ps1956 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1521196&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Peter (ps1956) Assigned to: Nobody/Anonymous (nobody) Summary: smtplib login fails with aol smtp server Initial Comment: Sending email to AOL's smtp server fails with "501 INVALID SASL FORMAT OR LENGTH". This patch allows the default order (AUTH_CRAM_MD5, AUTH_PLAIN, AUTH_LOGIN) used to select the AUTH method to be overridden with a new optional preferred_auths parameter. By changing the order so that AUTH_LOGIN comes before AUTH_PLAIN, AOL's smtp server is happy. This patch may be useful for other stmp servers that fail on login when using the default AUTH selection order. Sample usage: if aol: auths = [smtplib.AUTH_CRAM_MD5, smtplib.AUTH_LOGIN, smtplib.AUTH_PLAIN] smtp.login(fromUser.split("@")[0], passwd, auths) else: smtp.login(fromUser, passwd) I tested the patch using python 2.4.3 on slackware 10.2 (kernel 2.4.32). ---------------------------------------------------------------------- >Comment By: Peter (ps1956) Date: 2007-04-04 06:31 Message: Logged In: YES user_id=1554321 Originator: YES File Added: smtplib.patch ---------------------------------------------------------------------- Comment By: Stephen Hansen (aptshansen) Date: 2007-03-16 02:21 Message: Logged In: YES user_id=1720594 Originator: NO This seems reasonable to me; the preference of which authentication modes to use shouldn't be a hardcoded decision in smtplib, especially if one of the biggest email servers around breaks if we can't. However, I think it needs a couple things done before it should be accepted: #1) The code patch itself needs to be done against the trunk, as it doesn't currently apply cleanly (it's a trivial fix) #2) The documentation patch should include mention of the "constants" of smtplib.AUTH_* ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1521196&group_id=5470 From noreply at sourceforge.net Wed Apr 4 20:09:39 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 11:09:39 -0700 Subject: [Patches] [ python-Patches-1273829 ] extending os.walk to support following symlinks Message-ID: Patches item #1273829, was opened at 2005-08-26 09:21 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: Accepted Priority: 5 Private: No Submitted By: Erick Tryzelaar (idadesub) >Assigned to: Ziga Seilnacht (zseil) Summary: extending os.walk to support following symlinks Initial Comment: this is a very simple extension to os.walk to optionally walk down into symlink directories, when it is supported by the operating system. This patch keeps a compatible interface by having this option disabled. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-04 20:09 Message: Logged In: YES user_id=21627 Originator: NO The patch is fine; please apply. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-03 08:27 Message: Logged In: YES user_id=1326842 Originator: NO Here is a patch that fixes test_os.WalkTests on Windows. I also added a tearDown method to WalkTests and removed Collin's changes to MakedirTests, because they are not needed anymore. File Added: test_os.diff ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-23 16:31 Message: Logged In: YES user_id=21627 Originator: NO The test suite patch (54482) broke the windows build. The assertion self.assertEqual(len(all), 4) fails because it finds 5 directories (link is a directory), then the rest of the code is not executed. In turn, the tear-down code is not run, leaving the directories behind. This causes subsequent tests to fail. When fixing the code, I recommend to move the tear-down code into the tearDown method of the test, to make sure it is always run. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-16 09:22 Message: Logged In: YES user_id=849994 Originator: NO Added docs and tests and committed in rev. 54407. ---------------------------------------------------------------------- Comment By: Stephen Hansen (aptshansen) Date: 2007-03-16 08:34 Message: Logged In: YES user_id=1720594 Originator: NO I'm not sure if this is a good idea; although I am certain it would be useful at times. Dangerous! But if its documented, ... and there's our problem :) Patch applies cleanly to the HEAD, but needs documentation. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-08-26 10:23 Message: Logged In: YES user_id=1188172 The docs to os.walk say: """ Note: On systems that support symbolic links, links to subdirectories appear in dirnames lists, but walk() will not visit them (infinite loops are hard to avoid when following symbolic links). To visit linked directories, you can identify them with os.path.islink(path), and invoke walk(path) on each directly. """ Nevertheless, we could offer this option with a caution message. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 From noreply at sourceforge.net Wed Apr 4 20:41:03 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 11:41:03 -0700 Subject: [Patches] [ python-Patches-1273829 ] extending os.walk to support following symlinks Message-ID: Patches item #1273829, was opened at 2005-08-26 09:21 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Erick Tryzelaar (idadesub) Assigned to: Ziga Seilnacht (zseil) Summary: extending os.walk to support following symlinks Initial Comment: this is a very simple extension to os.walk to optionally walk down into symlink directories, when it is supported by the operating system. This patch keeps a compatible interface by having this option disabled. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-04 20:41 Message: Logged In: YES user_id=1326842 Originator: NO Commited as revision 54689. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-04 20:09 Message: Logged In: YES user_id=21627 Originator: NO The patch is fine; please apply. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-03 08:27 Message: Logged In: YES user_id=1326842 Originator: NO Here is a patch that fixes test_os.WalkTests on Windows. I also added a tearDown method to WalkTests and removed Collin's changes to MakedirTests, because they are not needed anymore. File Added: test_os.diff ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-23 16:31 Message: Logged In: YES user_id=21627 Originator: NO The test suite patch (54482) broke the windows build. The assertion self.assertEqual(len(all), 4) fails because it finds 5 directories (link is a directory), then the rest of the code is not executed. In turn, the tear-down code is not run, leaving the directories behind. This causes subsequent tests to fail. When fixing the code, I recommend to move the tear-down code into the tearDown method of the test, to make sure it is always run. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-16 09:22 Message: Logged In: YES user_id=849994 Originator: NO Added docs and tests and committed in rev. 54407. ---------------------------------------------------------------------- Comment By: Stephen Hansen (aptshansen) Date: 2007-03-16 08:34 Message: Logged In: YES user_id=1720594 Originator: NO I'm not sure if this is a good idea; although I am certain it would be useful at times. Dangerous! But if its documented, ... and there's our problem :) Patch applies cleanly to the HEAD, but needs documentation. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-08-26 10:23 Message: Logged In: YES user_id=1188172 The docs to os.walk say: """ Note: On systems that support symbolic links, links to subdirectories appear in dirnames lists, but walk() will not visit them (infinite loops are hard to avoid when following symbolic links). To visit linked directories, you can identify them with os.path.islink(path), and invoke walk(path) on each directly. """ Nevertheless, we could offer this option with a caution message. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 From noreply at sourceforge.net Wed Apr 4 20:47:01 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 11:47:01 -0700 Subject: [Patches] [ python-Patches-1310381 ] httplib : SSL fake sockets to close correctly Message-ID: Patches item #1310381, was opened at 2005-10-01 19:21 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1310381&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Out of Date Priority: 5 Private: No Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: httplib : SSL fake sockets to close correctly Initial Comment: [refers to bug #978833 ; ] Now I managed to solve the problem for me with attached patch of httplib.py: a explicit shutdown ( 2 or 1 ) of the (faked) ssl'ed socket solves it definitely. I still guess the ssl'ed socket (ssl dll) should do that auto on sock.close() ; there may be still a bug in ssl.c . the original problem obviously only shows up with FTPS connection (as described in the bug), which relies on correct socket closing and also shows with certain https firewalls/caches which cause a sslerror: (8, 'EOF occurred in violation of protocol') . *** \usr\src\py24old/httplib.py Sat Sep 24 21:35:28 2005 --- httplib.py Sat Sep 24 21:37:48 2005 *************** class SharedSocket: *** 899,904 **** --- 899,905 ---- self._refcnt -= 1 assert self._refcnt >= 0 if self._refcnt == 0: + self.sock.shutdown(2) self.sock.close() def __del__(self): ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-04 20:47 Message: Logged In: YES user_id=21627 Originator: NO The original problem is now being solved in a different way; rejecting this patch as out-of-date ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1310381&group_id=5470 From noreply at sourceforge.net Wed Apr 4 23:28:18 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 14:28:18 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 23:23 Message generated for change (Comment added) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 15:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 01:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Wed Apr 4 23:59:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 14:59:12 -0700 Subject: [Patches] [ python-Patches-1273829 ] extending os.walk to support following symlinks Message-ID: Patches item #1273829, was opened at 2005-08-26 07:21 Message generated for change (Comment added) made by tmick You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Erick Tryzelaar (idadesub) Assigned to: Ziga Seilnacht (zseil) Summary: extending os.walk to support following symlinks Initial Comment: this is a very simple extension to os.walk to optionally walk down into symlink directories, when it is supported by the operating system. This patch keeps a compatible interface by having this option disabled. ---------------------------------------------------------------------- >Comment By: Trent Mick (tmick) Date: 2007-04-04 21:59 Message: Logged In: YES user_id=34892 Originator: NO Is there a separate bug for doc updates for this? ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-04 18:41 Message: Logged In: YES user_id=1326842 Originator: NO Commited as revision 54689. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-04 18:09 Message: Logged In: YES user_id=21627 Originator: NO The patch is fine; please apply. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-03 06:27 Message: Logged In: YES user_id=1326842 Originator: NO Here is a patch that fixes test_os.WalkTests on Windows. I also added a tearDown method to WalkTests and removed Collin's changes to MakedirTests, because they are not needed anymore. File Added: test_os.diff ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-23 15:31 Message: Logged In: YES user_id=21627 Originator: NO The test suite patch (54482) broke the windows build. The assertion self.assertEqual(len(all), 4) fails because it finds 5 directories (link is a directory), then the rest of the code is not executed. In turn, the tear-down code is not run, leaving the directories behind. This causes subsequent tests to fail. When fixing the code, I recommend to move the tear-down code into the tearDown method of the test, to make sure it is always run. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-16 08:22 Message: Logged In: YES user_id=849994 Originator: NO Added docs and tests and committed in rev. 54407. ---------------------------------------------------------------------- Comment By: Stephen Hansen (aptshansen) Date: 2007-03-16 07:34 Message: Logged In: YES user_id=1720594 Originator: NO I'm not sure if this is a good idea; although I am certain it would be useful at times. Dangerous! But if its documented, ... and there's our problem :) Patch applies cleanly to the HEAD, but needs documentation. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-08-26 08:23 Message: Logged In: YES user_id=1188172 The docs to os.walk say: """ Note: On systems that support symbolic links, links to subdirectories appear in dirnames lists, but walk() will not visit them (infinite loops are hard to avoid when following symbolic links). To visit linked directories, you can identify them with os.path.islink(path), and invoke walk(path) on each directly. """ Nevertheless, we could offer this option with a caution message. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 From noreply at sourceforge.net Thu Apr 5 04:01:50 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 04 Apr 2007 19:01:50 -0700 Subject: [Patches] [ python-Patches-1691032 ] Migrate test_minidom.py to unittest Message-ID: Patches item #1691032, was opened at 2007-03-30 00:15 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: Migrate test_minidom.py to unittest Initial Comment: Update Lib/test/test_minidom.py to use the unittest module. Also: I noticed the test_minidom.py contains some code (now long dead) to check for reference cycles. The code is active only when Node.allnodes exists; this hasn't existed for over four years now. After re-implementing that test using gc.DEBUG_SAVEALL, I found that many tests failed because of missing unlink() calls. I added them. After that, one test was still failing. This is because expatbuilder.parseString() and friends do not clean up some cyclic references on error. The patch includes a fix for that (in Lib/xml/dom/expatbuilder.py). An alternative would be to ditch the cycle-checking test entirely. I can submit a patch that does that instead, if desired. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-04 22:01 Message: Logged In: YES user_id=18139 Originator: YES Collin: Thanks for taking the time to look at this. I don't think this change is worth making. I've started working on some new features in minidom and it seems like every time I exercise a new piece of code, the gc finds a small number of objects in cycles. For example, parseFragmentString() seems to leave a few objects (even with expatbuilder.second.patch). We're just going to have to live with it-- I don't think anyone has time or desire to hunt down them all down. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-01 17:18 Message: Logged In: YES user_id=18139 Originator: YES File Added: expatbuilder.second.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-01 17:18 Message: Logged In: YES user_id=18139 Originator: YES OK, I've attached two new patches. test_minidom.second.patch - Changes "confirm" to "assert_" or "assertEquals", breaks multi-line assertions into many one-line assertions, adds convenience methods "assertSameNode" and "assertNotSameNode", etc. Also replaces the Node.allnodes test with gc-based test and adds a few calls to unlink(). expatbuilder.second.patch - Fixes the test failure discovered by the new gc testing. File Added: test_minidom.second.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-30 02:49 Message: Logged In: YES user_id=1344176 Originator: NO test_minidom already uses unittest (as of patch #1683397, SVN r54603). Could you update your patch to the latest trunk revision? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 From noreply at sourceforge.net Thu Apr 5 18:14:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 05 Apr 2007 09:14:23 -0700 Subject: [Patches] [ python-Patches-1693258 ] Fix for duplicate "preferences" menu-OS X Message-ID: Patches item #1693258, was opened at 2007-04-02 17:15 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1693258&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Walzer (wordtech) >Assigned to: Ronald Oussoren (ronaldoussoren) Summary: Fix for duplicate "preferences" menu-OS X Initial Comment: Version 8.4.14 of Tcl/Tk Aqua (for OS X) hard-codes a "Preferences" item in the Apple menu. As a result, IDLE now has two "Preferences" items--the active one that is coded by default, and an inactive item that is hard-coded by Tk. The attached patch to macosxSupport.py checks the installed version of Tk, and if it is 8.4.14, substitutes the hard-coded Tk "preferences" menu and binds IDLE's "preferences" dialog to this menu item. If Tk is version 8.4.13 or less, leave the menu alone. This patch addresses Python bug #1691411 opened at 2007-03-30. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1693258&group_id=5470 From noreply at sourceforge.net Thu Apr 5 20:26:28 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 05 Apr 2007 11:26:28 -0700 Subject: [Patches] [ python-Patches-1686273 ] More PEP 3116 classes Message-ID: Patches item #1686273, was opened at 2007-03-22 14:18 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686273&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mark Russell (mark_t_russell) Assigned to: Nobody/Anonymous (nobody) Summary: More PEP 3116 classes Initial Comment: More PEP 3116 implementation: - docstrings for RawIOBase methods - Simple implementations of StringIO, TextIOBase and TextIOWrapper - Simplify while condition in BufferedReader.read() - Add unit tests for StringIO (modified copy of the ByteIO tests) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-05 14:26 Message: Logged In: YES user_id=6380 Originator: NO I'm finally getting to this. Sorry for my tardiness! I've started by applying the PEP patch. Now reviewing the code patch... ---------------------------------------------------------------------- Comment By: Mark Russell (mark_t_russell) Date: 2007-03-24 18:46 Message: Logged In: YES user_id=1017234 Originator: YES File Added: pep-3116-pep.diff ---------------------------------------------------------------------- Comment By: Mark Russell (mark_t_russell) Date: 2007-03-24 18:34 Message: Logged In: YES user_id=1017234 Originator: YES More changes: - Make TextIOWrapper.readline() reasonably efficient - Fix TextIOWrapper.read() - Unit tests for TextIOWrapper .readline() and .read() (more needed) - Readahead support for BufferedReader (i.e. so it reads in buffer size chunks) - Fix codecs.BufferedIncrementalDecoder buffer type (bytes not str) - Unit test to check the raw reads that BufferedReader is doing - Fix BufferedRandom positioning bug when switching from reading to writing File Added: pep-3116-textio.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686273&group_id=5470 From noreply at sourceforge.net Thu Apr 5 21:49:15 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 05 Apr 2007 12:49:15 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Thu Apr 5 22:26:19 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 05 Apr 2007 13:26:19 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 15:49 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 16:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 11:52:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 02:52:23 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by arve_knudsen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 11:59:27 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 02:59:27 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by arve_knudsen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 12:09:25 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 03:09:25 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) >Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 12:23:37 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 03:23:37 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by arve_knudsen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 19:34:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 10:34:33 -0700 Subject: [Patches] [ python-Patches-1686273 ] More PEP 3116 classes Message-ID: Patches item #1686273, was opened at 2007-03-22 14:18 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686273&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Mark Russell (mark_t_russell) Assigned to: Nobody/Anonymous (nobody) Summary: More PEP 3116 classes Initial Comment: More PEP 3116 implementation: - docstrings for RawIOBase methods - Simple implementations of StringIO, TextIOBase and TextIOWrapper - Simplify while condition in BufferedReader.read() - Add unit tests for StringIO (modified copy of the ByteIO tests) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-06 13:34 Message: Logged In: YES user_id=6380 Originator: NO The code patches are checked in too, with some cleanup applied. I rolled back your change to codecs.py, as it broke test_codecs.py; instead, I added an ugly hack where we patch the codec after it's been created. I'll be working more on this code soon, I expect. Stay tuned -- and thanks for your work! (Technically, the PSF needs a contribution form from you, see http://www.python.org/psf/contrib/.) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-05 14:26 Message: Logged In: YES user_id=6380 Originator: NO I'm finally getting to this. Sorry for my tardiness! I've started by applying the PEP patch. Now reviewing the code patch... ---------------------------------------------------------------------- Comment By: Mark Russell (mark_t_russell) Date: 2007-03-24 18:46 Message: Logged In: YES user_id=1017234 Originator: YES File Added: pep-3116-pep.diff ---------------------------------------------------------------------- Comment By: Mark Russell (mark_t_russell) Date: 2007-03-24 18:34 Message: Logged In: YES user_id=1017234 Originator: YES More changes: - Make TextIOWrapper.readline() reasonably efficient - Fix TextIOWrapper.read() - Unit tests for TextIOWrapper .readline() and .read() (more needed) - Readahead support for BufferedReader (i.e. so it reads in buffer size chunks) - Fix codecs.BufferedIncrementalDecoder buffer type (bytes not str) - Unit test to check the raw reads that BufferedReader is doing - Fix BufferedRandom positioning bug when switching from reading to writing File Added: pep-3116-textio.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686273&group_id=5470 From noreply at sourceforge.net Fri Apr 6 20:03:28 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 11:03:28 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 20:55:37 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 11:55:37 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 15:49 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 14:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 14:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 06:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 06:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 05:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 05:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 16:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 22:14:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 13:14:23 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 6 23:41:41 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 14:41:41 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 15:49 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 17:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 16:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 14:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 14:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 06:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 06:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 05:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 05:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 16:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Sat Apr 7 00:35:02 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 15:35:02 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by arve_knudsen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 00:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 23:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Sat Apr 7 01:03:13 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 16:03:13 -0700 Subject: [Patches] [ python-Patches-1695862 ] Fix test_urllib on Windows buildbots Message-ID: Patches item #1695862, was opened at 2007-04-07 01:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ziga Seilnacht (zseil) Assigned to: Nobody/Anonymous (nobody) Summary: Fix test_urllib on Windows buildbots Initial Comment: test_urllib is currently failing on Windows buildbots because test_os at one time created a directory named test_support.TESTFN + ".2". test_os has been fixed, but the old directory is still present on buildbots. This patch is a one time fix that should be removed after the first succesful buildbot run. I added the cleanup code to test_urllib, because buildbots run tests in random order. The other option is to ask the buildbot slave maintainers to manually remove the offending directory. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 From noreply at sourceforge.net Sat Apr 7 01:06:59 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 16:06:59 -0700 Subject: [Patches] [ python-Patches-1695862 ] Fix test_urllib on Windows buildbots Message-ID: Patches item #1695862, was opened at 2007-04-07 01:03 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ziga Seilnacht (zseil) Assigned to: Nobody/Anonymous (nobody) Summary: Fix test_urllib on Windows buildbots Initial Comment: test_urllib is currently failing on Windows buildbots because test_os at one time created a directory named test_support.TESTFN + ".2". test_os has been fixed, but the old directory is still present on buildbots. This patch is a one time fix that should be removed after the first succesful buildbot run. I added the cleanup code to test_urllib, because buildbots run tests in random order. The other option is to ask the buildbot slave maintainers to manually remove the offending directory. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-07 01:06 Message: Logged In: YES user_id=1326842 Originator: YES Here is the second patch that should be applied once the first one has done its job. It also removes Neal's cleanups, because they are not needed anymore. File Added: cleanup_cleanup.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 From noreply at sourceforge.net Sat Apr 7 02:10:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 06 Apr 2007 17:10:33 -0700 Subject: [Patches] [ python-Patches-1273829 ] extending os.walk to support following symlinks Message-ID: Patches item #1273829, was opened at 2005-08-26 09:21 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Erick Tryzelaar (idadesub) Assigned to: Ziga Seilnacht (zseil) Summary: extending os.walk to support following symlinks Initial Comment: this is a very simple extension to os.walk to optionally walk down into symlink directories, when it is supported by the operating system. This patch keeps a compatible interface by having this option disabled. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-07 02:10 Message: Logged In: YES user_id=1326842 Originator: NO Hi Trent, There is no separate issue yet. What is missing in the docs? Georg updated the docs when he checked in this patch. ---------------------------------------------------------------------- Comment By: Trent Mick (tmick) Date: 2007-04-04 23:59 Message: Logged In: YES user_id=34892 Originator: NO Is there a separate bug for doc updates for this? ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-04 20:41 Message: Logged In: YES user_id=1326842 Originator: NO Commited as revision 54689. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-04 20:09 Message: Logged In: YES user_id=21627 Originator: NO The patch is fine; please apply. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-03 08:27 Message: Logged In: YES user_id=1326842 Originator: NO Here is a patch that fixes test_os.WalkTests on Windows. I also added a tearDown method to WalkTests and removed Collin's changes to MakedirTests, because they are not needed anymore. File Added: test_os.diff ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-23 16:31 Message: Logged In: YES user_id=21627 Originator: NO The test suite patch (54482) broke the windows build. The assertion self.assertEqual(len(all), 4) fails because it finds 5 directories (link is a directory), then the rest of the code is not executed. In turn, the tear-down code is not run, leaving the directories behind. This causes subsequent tests to fail. When fixing the code, I recommend to move the tear-down code into the tearDown method of the test, to make sure it is always run. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-16 09:22 Message: Logged In: YES user_id=849994 Originator: NO Added docs and tests and committed in rev. 54407. ---------------------------------------------------------------------- Comment By: Stephen Hansen (aptshansen) Date: 2007-03-16 08:34 Message: Logged In: YES user_id=1720594 Originator: NO I'm not sure if this is a good idea; although I am certain it would be useful at times. Dangerous! But if its documented, ... and there's our problem :) Patch applies cleanly to the HEAD, but needs documentation. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-08-26 10:23 Message: Logged In: YES user_id=1188172 The docs to os.walk say: """ Note: On systems that support symbolic links, links to subdirectories appear in dirnames lists, but walk() will not visit them (infinite loops are hard to avoid when following symbolic links). To visit linked directories, you can identify them with os.path.islink(path), and invoke walk(path) on each directly. """ Nevertheless, we could offer this option with a caution message. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 From Edavida2 at aol.com Sat Apr 7 09:59:00 2007 From: Edavida2 at aol.com (Edavida2 at aol.com) Date: Sat, 7 Apr 2007 03:59:00 EDT Subject: [Patches] Old lady gets ass fucked really hard!...portent Message-ID: HI THEREmck_emall at yahoo.com ************************************** See what's free at http://www.aol.com. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20070407/4ebf4446/attachment.html From noreply at sourceforge.net Sat Apr 7 11:33:02 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 07 Apr 2007 02:33:02 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-07 11:33 Message: Logged In: YES user_id=642936 Originator: NO Thank you for making clear that this is a regression. This is a completely different thing and makes the previous discussion obsolete... Unfortunately, this will not go into 2.5.1, we're too late. But I will come up with a reasonable fix and testcases for 2.5.2. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 00:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 23:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Sat Apr 7 15:57:37 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 07 Apr 2007 06:57:37 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by arve_knudsen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 15:57 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-07 11:33 Message: Logged In: YES user_id=642936 Originator: NO Thank you for making clear that this is a regression. This is a completely different thing and makes the previous discussion obsolete... Unfortunately, this will not go into 2.5.1, we're too late. But I will come up with a reasonable fix and testcases for 2.5.2. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 00:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 23:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Sat Apr 7 16:07:34 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 07 Apr 2007 07:07:34 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by arve_knudsen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 16:07 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 15:57 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-07 11:33 Message: Logged In: YES user_id=642936 Originator: NO Thank you for making clear that this is a regression. This is a completely different thing and makes the previous discussion obsolete... Unfortunately, this will not go into 2.5.1, we're too late. But I will come up with a reasonable fix and testcases for 2.5.2. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 00:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 23:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Sat Apr 7 23:39:35 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 07 Apr 2007 14:39:35 -0700 Subject: [Patches] [ python-Patches-1683908 ] PEP 361 Warnings Message-ID: Patches item #1683908, was opened at 2007-03-19 13:32 Message generated for change (Comment added) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1683908&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Thomas Wouters (twouters) Summary: PEP 361 Warnings Initial Comment: Attached is a patch that adds some of the warnings mentioned in PEP 361, specifically: * `repr` * cmp() and __cmp__ * apply() * coerce() and __coerce__ * reduce() * {}.has_key() I'm unsure what conclusions were reaches wrt py3k-specific command line options or warning categories, so I've used DeprecationWarning as a placeholder. Not surprisingly, this causes large amounts of noise in the test suite (but no failures). Thus far I've focused on warnings that (in my opinion) should be fixed in 2.x code. Many of the other warnings (such as using {}.iteritems()) are better handled using the 2to3 translator. Again though, I'm unsure if they will ultimately be given a distinct warning category, so I await input on that. ---------------------------------------------------------------------- >Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-07 15:39 Message: Logged In: YES user_id=12364 Originator: YES Fixes/changes * Includes Patch #1692664, for my own sanity. Hopefully won't cause a problem when applying. * Warnings for __coerce__/__cmp__ during class creation now show as from the functions themselves. Uses the class as a fallback if the methods are not real functions. * Leaks shown by regrtest have been fixed. * test_descr no longer fails, although the problem in p3yk presumably still exists. * __warningregistry__ is handled better, so (combined with Patch #1692664 and other changes) all the oddities are gone. * Some tests have occasionally failed with regrtest (such as test_tcl), but I don't think they're related to this patch. Outstanding issues * URL needs to point to an actual page * DeprecationWarning needs to be replaced with something more appropriate * "warning:" lines early during interpreter startup need to be silenced * warnings module needs to control _Py_Py3kWarningFlag File Added: python2.6-warnings3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-03-25 21:23 Message: Logged In: YES user_id=12364 Originator: YES Version 2 of my patch: * Drops the warning for ``. Anthony Baxter's patch (#1631035) handles this better. * __cmp__ and __coerce__ are checked for at class creation time as well as when called. Currently will emit two warnings. * A private function (_warnpy3k()) in warnings.py was added to take care of extracting warning location from the passed in function, but I've been unable to get that working properly. At the moment it only serves as a single location (or two locations, if you include the C wrapper) to modify the warning message. * The warning message was modified to be more informative, including a URL. The URL currently points nowhere. * A global flag (_Py_Py3kWarningFlag) was added to skip the warning code when not enabled. I have located it with the other C interfaces to the warnings module on the assumption that it will be integrated as part of Neal's rewrite of warnings.py in C. It defaults to on and I have not included a command line option to control it. * Some warnings are omitted before warnings.py gets loaded, causing them to always be printed. Presumably this will be fixed when warnings.py is rewritten in C. * test_subprocess fails. This is due warnings in the output and will be fixed when the warnings default to silent. * test_descr fails. This is due to a test (line 1298, SF bug 572567) that defines a class with __cmp__ and counts objects using gc.get_objects(). Although easily fixed by switching to __eq__, I've noticed that the p3yk branch uses __cmp__ as well. It may be failing silently? * One of the tests fail if I merge together the two loops it contains, but it shouldn't. I'm guessing this is due to not registering the warnings in the right location, which is what _warnpy3k() should fix. * Running regrtest on test_py3kwarnings.py fails. I'm guessing this is due to the above. * Running regrtest on everything shows many leaks (and a few more failures). I'm hoping this is due to the above as well. File Added: python2.6-warnings2.diff ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2007-03-20 22:02 Message: Logged In: YES user_id=29957 Originator: NO Here's a -W py3k patch I did. Think it went to the list, not the trackers. File Added: warn3.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-03-20 21:53 Message: Logged In: YES user_id=33168 Originator: NO Oooh, Anthony pointed me to his patch: http://python.org/sf/1631035 So no need to impl anything that's in there. I don't think there's much overlap so far, but I didn't look that closely. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-03-20 21:48 Message: Logged In: YES user_id=33168 Originator: NO Thomas, should there be a 3k warning category that's a subclass of DeprecationWarning? Should this patch impl the -3k flag or whatever it's called? Damn, I really gotta get that warnings patch done. First I gotta get the buildbots in decent shape, get 2.5.1 out, then xrange patch, then warnings. That list sucks. It doesn't even include crap like get inbox less than 200 unread msgs or do taxes. :-( ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-03-20 16:32 Message: Logged In: YES user_id=12364 Originator: YES I originally had the `` warning emitted during the parse stage, but this prevents it from showing when the .pyc file already exists. Which is more important? And is it an actual DeprecationWarning or a PendingDeprecationWarning? I didn't notice a way to produce a warning at both definition time and use time, without emitting it twice. I agree that both is better, so I'll figure it out. Linking to a wiki page would be far better than trying to squeeze a full explanation into one sentence. ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2007-03-20 15:57 Message: Logged In: YES user_id=34209 Originator: NO The ``-warning should be emitted by the parser, not when the UNARY_CONVERT opcode is executed. I'm still not entirely sure __cmp__/cmp() should be deprecated (cmp() is terribly convenient, and I haven't heard Guido say they really should be. Maybe just __cmp__ should go.) I wonder if we shouldn't do the warning for __coerce__ and __cmp__ at class creation time as well as at call time. (Class creation is earlier and oftner, but doesn't catch all cases. The warnings registry would make sure the warning is only emitted once, by default.) Some of the messages could maybe be more informative; perhaps we can link to a wiki page with py3k-deprecation info. On the other hand, people will only see most of these if they do the -Wpy3k thing. (The ``-warning should be a real deprecation warning in the parser, not related to -Wpy3k.) Other than those minor points, looks good. I think it should wait for the C-warnings-module Neal N's working on, though, before we really check it in. Assigning to Neal for him to take a look at. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1683908&group_id=5470 From noreply at sourceforge.net Sun Apr 8 13:44:55 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 08 Apr 2007 04:44:55 -0700 Subject: [Patches] [ python-Patches-850482 ] Enhance frame handing in warnings.warn() Message-ID: Patches item #850482, was opened at 2003-11-28 00:12 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=850482&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Enhance frame handing in warnings.warn() Initial Comment: This patch enhances warnings.warn() in the following way: The stacklevel passed in may be negative. In that case the call stack is searched for the innermost frame whose module name differs in the first -stacklevel components. This frame will be used in the report. So when you have the following call stack: m1.f() m1.m11.f() m1.m12.f() m2.m21.f() m2.m22.f() and the innermost function() m2.m22.f() call warnings.warn() with a stacklevel of -2 the frame reported will be from m1.m12.f(), because it is the first one from outside the m2 package. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2007-04-08 13:44 Message: Logged In: YES user_id=89016 Originator: YES Here's an updated patch that implements your suggestion: skipmodules is a regular expression. Any module whose name matches the RE will be skipped when determining the stacklevel. File Added: diff2.txt ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-02 10:41 Message: Logged In: YES user_id=1326842 Originator: NO I don't know how desireable this feature is, but I think that a better solution would be to add a new parameter to warnings.warn(), instead of reusing the old one. Also, your description is wrong; when warn() in your example is called with stacklevel -2, the reported frame is m2.m21.f(). To get your result, you have to call warn() with stacklevel -1. I think that a better solution would be to add a skipmodules='moduleprefix' parameter to the warn() function. warn() would then simply look for the first frame whose module name doesn't start with this prefix. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=850482&group_id=5470 From noreply at sourceforge.net Sun Apr 8 14:03:06 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 08 Apr 2007 05:03:06 -0700 Subject: [Patches] [ python-Patches-1696393 ] Remove redundant code in ntpath.walk() Message-ID: Patches item #1696393, was opened at 2007-04-08 14:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696393&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Haggerty (mhagger) Assigned to: Nobody/Anonymous (nobody) Summary: Remove redundant code in ntpath.walk() Initial Comment: The documentation for os.listdir() explicitly says that it omits directories '.' and '..' even if they are present in the directory, and the implementation in posixmodule.c seems to agree. Yet the implementation of os.path.walk() in ntpath.py still checks for and excludes '.' and '..' from the names gotten from os.listdir(). This seems like dead code. The analogous check was removed from posixpath.py by Guido in r14534, with the comment > Remove some redundant logic from walk() -- there's no need to check > for "." and "..", since listdir() no longer returns those. The attached patch removes the extra check. It is relative to trunk r54712. I don't have a Windows computer around to test this patch, but it seems straightforward enough. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696393&group_id=5470 From noreply at sourceforge.net Sun Apr 8 16:35:47 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 08 Apr 2007 07:35:47 -0700 Subject: [Patches] [ python-Patches-1696444 ] Adding an index method to tuples Message-ID: Patches item #1696444, was opened at 2007-04-08 16:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Adding an index method to tuples Initial Comment: This is just a simple copy and paste from the list implementation - not exactly the most elegant solution, but perhaps a demonstration that technical limitations aren't the reason why the tuple doesn't have an index method. I suppose a common sequence function library would be best if such a feature were eventually deemed compelling. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 From noreply at sourceforge.net Mon Apr 9 18:07:57 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 09:07:57 -0700 Subject: [Patches] [ python-Patches-1273829 ] extending os.walk to support following symlinks Message-ID: Patches item #1273829, was opened at 2005-08-26 07:21 Message generated for change (Comment added) made by tmick You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Erick Tryzelaar (idadesub) Assigned to: Ziga Seilnacht (zseil) Summary: extending os.walk to support following symlinks Initial Comment: this is a very simple extension to os.walk to optionally walk down into symlink directories, when it is supported by the operating system. This patch keeps a compatible interface by having this option disabled. ---------------------------------------------------------------------- >Comment By: Trent Mick (tmick) Date: 2007-04-09 16:07 Message: Logged In: YES user_id=34892 Originator: NO Ziga, My mistake: I failed to notice the change to Doc/lib/libos.tex. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-07 00:10 Message: Logged In: YES user_id=1326842 Originator: NO Hi Trent, There is no separate issue yet. What is missing in the docs? Georg updated the docs when he checked in this patch. ---------------------------------------------------------------------- Comment By: Trent Mick (tmick) Date: 2007-04-04 21:59 Message: Logged In: YES user_id=34892 Originator: NO Is there a separate bug for doc updates for this? ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-04 18:41 Message: Logged In: YES user_id=1326842 Originator: NO Commited as revision 54689. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-04 18:09 Message: Logged In: YES user_id=21627 Originator: NO The patch is fine; please apply. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-03 06:27 Message: Logged In: YES user_id=1326842 Originator: NO Here is a patch that fixes test_os.WalkTests on Windows. I also added a tearDown method to WalkTests and removed Collin's changes to MakedirTests, because they are not needed anymore. File Added: test_os.diff ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-03-23 15:31 Message: Logged In: YES user_id=21627 Originator: NO The test suite patch (54482) broke the windows build. The assertion self.assertEqual(len(all), 4) fails because it finds 5 directories (link is a directory), then the rest of the code is not executed. In turn, the tear-down code is not run, leaving the directories behind. This causes subsequent tests to fail. When fixing the code, I recommend to move the tear-down code into the tearDown method of the test, to make sure it is always run. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-16 08:22 Message: Logged In: YES user_id=849994 Originator: NO Added docs and tests and committed in rev. 54407. ---------------------------------------------------------------------- Comment By: Stephen Hansen (aptshansen) Date: 2007-03-16 07:34 Message: Logged In: YES user_id=1720594 Originator: NO I'm not sure if this is a good idea; although I am certain it would be useful at times. Dangerous! But if its documented, ... and there's our problem :) Patch applies cleanly to the HEAD, but needs documentation. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-08-26 08:23 Message: Logged In: YES user_id=1188172 The docs to os.walk say: """ Note: On systems that support symbolic links, links to subdirectories appear in dirnames lists, but walk() will not visit them (infinite loops are hard to avoid when following symbolic links). To visit linked directories, you can identify them with os.path.islink(path), and invoke walk(path) on each directly. """ Nevertheless, we could offer this option with a caution message. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1273829&group_id=5470 From noreply at sourceforge.net Mon Apr 9 18:24:48 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 09:24:48 -0700 Subject: [Patches] [ python-Patches-1696444 ] Adding an index method to tuples Message-ID: Patches item #1696444, was opened at 2007-04-08 07:35 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Adding an index method to tuples Initial Comment: This is just a simple copy and paste from the list implementation - not exactly the most elegant solution, but perhaps a demonstration that technical limitations aren't the reason why the tuple doesn't have an index method. I suppose a common sequence function library would be best if such a feature were eventually deemed compelling. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 09:24 Message: Logged In: YES user_id=341410 Originator: NO Adding an .index method to tuples suggests that a tuple is more or less an immutable list. This is not the case. Typically tuples are used to hold disparate pieces of information of different semantics and types, where as a list typically contains a list of a single type with identical meaning. For examples we merely need to look at the outputs of str.partition and str.split . Suggested close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 From noreply at sourceforge.net Mon Apr 9 19:08:07 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 10:08:07 -0700 Subject: [Patches] [ python-Patches-1696444 ] Adding an index method to tuples Message-ID: Patches item #1696444, was opened at 2007-04-08 09:35 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Adding an index method to tuples Initial Comment: This is just a simple copy and paste from the list implementation - not exactly the most elegant solution, but perhaps a demonstration that technical limitations aren't the reason why the tuple doesn't have an index method. I suppose a common sequence function library would be best if such a feature were eventually deemed compelling. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 12:08 Message: Logged In: YES user_id=80475 Originator: NO FWIW, I am no longer satisfied with the pat answer that "tuples are not immutable lists" or the "lists for for homogenous data, tuples are for hetrogenous data". Whether tuples should grow index() and count() methods should depend solely on whether real use cases arise often enough to warrant the new methods being added. Another consideration is that this feature request and discussion comes up so often that it may be worthwhile just to give-in to conserve our limited resources of time and brain-power. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 11:24 Message: Logged In: YES user_id=341410 Originator: NO Adding an .index method to tuples suggests that a tuple is more or less an immutable list. This is not the case. Typically tuples are used to hold disparate pieces of information of different semantics and types, where as a list typically contains a list of a single type with identical meaning. For examples we merely need to look at the outputs of str.partition and str.split . Suggested close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 From noreply at sourceforge.net Mon Apr 9 20:10:08 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 11:10:08 -0700 Subject: [Patches] [ python-Patches-1695862 ] Fix test_urllib on Windows buildbots Message-ID: Patches item #1695862, was opened at 2007-04-06 16:03 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ziga Seilnacht (zseil) >Assigned to: Ziga Seilnacht (zseil) Summary: Fix test_urllib on Windows buildbots Initial Comment: test_urllib is currently failing on Windows buildbots because test_os at one time created a directory named test_support.TESTFN + ".2". test_os has been fixed, but the old directory is still present on buildbots. This patch is a one time fix that should be removed after the first succesful buildbot run. I added the cleanup code to test_urllib, because buildbots run tests in random order. The other option is to ask the buildbot slave maintainers to manually remove the offending directory. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-09 11:10 Message: Logged In: YES user_id=33168 Originator: NO Ziga, if this passes on windows, please apply to trunk. 2.5 branch is stilly frozen (I forget if this is trunk only). Thanks for backing out my changes. They never did much. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-06 16:06 Message: Logged In: YES user_id=1326842 Originator: YES Here is the second patch that should be applied once the first one has done its job. It also removes Neal's cleanups, because they are not needed anymore. File Added: cleanup_cleanup.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 From noreply at sourceforge.net Mon Apr 9 21:51:41 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 12:51:41 -0700 Subject: [Patches] [ python-Patches-1696444 ] Adding an index method to tuples Message-ID: Patches item #1696444, was opened at 2007-04-08 16:35 Message generated for change (Comment added) made by pboddie You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Adding an index method to tuples Initial Comment: This is just a simple copy and paste from the list implementation - not exactly the most elegant solution, but perhaps a demonstration that technical limitations aren't the reason why the tuple doesn't have an index method. I suppose a common sequence function library would be best if such a feature were eventually deemed compelling. ---------------------------------------------------------------------- >Comment By: Paul Boddie (pboddie) Date: 2007-04-09 21:51 Message: Logged In: YES user_id=226443 Originator: YES See here for a recent extension of the discussion: http://groups.google.com/group/comp.lang.python/browse_frm/thread/7e9327a896c242e7/ For what it's worth, bypassing the dogma, methods like index and count on tuples are not exactly compelling in many situations (as I've pointed out before), but such issues should hardly derail the core development effort. If people consider such methods useful and if it diminishes perceived inelegance without method proliferation, just add them: people will be less frustrated and feel listened to; the language will become more regular. The only downside is any additional memory consumption. Objections related to what such methods mean, or what a tuple itself should be taken to mean, risk venturing into dogma and a consequent time-consuming discussion which takes 100 times the effort of just getting stuff done. Just look at the referenced thread (and the others of recent times) for example material. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 19:08 Message: Logged In: YES user_id=80475 Originator: NO FWIW, I am no longer satisfied with the pat answer that "tuples are not immutable lists" or the "lists for for homogenous data, tuples are for hetrogenous data". Whether tuples should grow index() and count() methods should depend solely on whether real use cases arise often enough to warrant the new methods being added. Another consideration is that this feature request and discussion comes up so often that it may be worthwhile just to give-in to conserve our limited resources of time and brain-power. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 18:24 Message: Logged In: YES user_id=341410 Originator: NO Adding an .index method to tuples suggests that a tuple is more or less an immutable list. This is not the case. Typically tuples are used to hold disparate pieces of information of different semantics and types, where as a list typically contains a list of a single type with identical meaning. For examples we merely need to look at the outputs of str.partition and str.split . Suggested close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 From noreply at sourceforge.net Mon Apr 9 22:06:37 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 13:06:37 -0700 Subject: [Patches] [ python-Patches-1696444 ] Adding an index method to tuples Message-ID: Patches item #1696444, was opened at 2007-04-08 07:35 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Adding an index method to tuples Initial Comment: This is just a simple copy and paste from the list implementation - not exactly the most elegant solution, but perhaps a demonstration that technical limitations aren't the reason why the tuple doesn't have an index method. I suppose a common sequence function library would be best if such a feature were eventually deemed compelling. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 13:06 Message: Logged In: YES user_id=341410 Originator: NO I wouldn't be averse to adding utility functions to some module where it would make sense. Generally, I would say collections (it has a deque and default dictionary already), though there is already a proposed constructor called 'count' that returns the counts of all items in a sequence as a default dictionary (requiring that the items all be hashable). The reason I'm not against utility functions is because then they would be usable for 3rd party non-list, non-tuple sequences, or even usable in cases like 'a = array.array(...); i = index(a, value)'. ---------------------------------------------------------------------- Comment By: Paul Boddie (pboddie) Date: 2007-04-09 12:51 Message: Logged In: YES user_id=226443 Originator: YES See here for a recent extension of the discussion: http://groups.google.com/group/comp.lang.python/browse_frm/thread/7e9327a896c242e7/ For what it's worth, bypassing the dogma, methods like index and count on tuples are not exactly compelling in many situations (as I've pointed out before), but such issues should hardly derail the core development effort. If people consider such methods useful and if it diminishes perceived inelegance without method proliferation, just add them: people will be less frustrated and feel listened to; the language will become more regular. The only downside is any additional memory consumption. Objections related to what such methods mean, or what a tuple itself should be taken to mean, risk venturing into dogma and a consequent time-consuming discussion which takes 100 times the effort of just getting stuff done. Just look at the referenced thread (and the others of recent times) for example material. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 10:08 Message: Logged In: YES user_id=80475 Originator: NO FWIW, I am no longer satisfied with the pat answer that "tuples are not immutable lists" or the "lists for for homogenous data, tuples are for hetrogenous data". Whether tuples should grow index() and count() methods should depend solely on whether real use cases arise often enough to warrant the new methods being added. Another consideration is that this feature request and discussion comes up so often that it may be worthwhile just to give-in to conserve our limited resources of time and brain-power. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 09:24 Message: Logged In: YES user_id=341410 Originator: NO Adding an .index method to tuples suggests that a tuple is more or less an immutable list. This is not the case. Typically tuples are used to hold disparate pieces of information of different semantics and types, where as a list typically contains a list of a single type with identical meaning. For examples we merely need to look at the outputs of str.partition and str.split . Suggested close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 From noreply at sourceforge.net Mon Apr 9 22:15:19 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 13:15:19 -0700 Subject: [Patches] [ python-Patches-1695862 ] Fix test_urllib on Windows buildbots Message-ID: Patches item #1695862, was opened at 2007-04-07 01:03 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ziga Seilnacht (zseil) Assigned to: Ziga Seilnacht (zseil) Summary: Fix test_urllib on Windows buildbots Initial Comment: test_urllib is currently failing on Windows buildbots because test_os at one time created a directory named test_support.TESTFN + ".2". test_os has been fixed, but the old directory is still present on buildbots. This patch is a one time fix that should be removed after the first succesful buildbot run. I added the cleanup code to test_urllib, because buildbots run tests in random order. The other option is to ask the buildbot slave maintainers to manually remove the offending directory. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-09 22:15 Message: Logged In: YES user_id=1326842 Originator: YES I checked in the first patch, and it fixed the problem on the Windows 2000 buildbot slave. I'm not entirely sure how the buildbot works, so I would like to wait with the second patch until the Windows XP slave comes back online. The test was failing only on the trunk, so there is no need to backport this change. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-09 20:10 Message: Logged In: YES user_id=33168 Originator: NO Ziga, if this passes on windows, please apply to trunk. 2.5 branch is stilly frozen (I forget if this is trunk only). Thanks for backing out my changes. They never did much. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-07 01:06 Message: Logged In: YES user_id=1326842 Originator: YES Here is the second patch that should be applied once the first one has done its job. It also removes Neal's cleanups, because they are not needed anymore. File Added: cleanup_cleanup.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 From noreply at sourceforge.net Mon Apr 9 22:36:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 13:36:33 -0700 Subject: [Patches] [ python-Patches-1690578 ] Don't block on Queue get/put when time is moved back Message-ID: Patches item #1690578, was opened at 2007-03-29 05:32 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: xiaowen (xiaowen) Assigned to: Nobody/Anonymous (nobody) Summary: Don't block on Queue get/put when time is moved back Initial Comment: When Queue get and put methods are called with a timeout, they effectively store the end time and might block until that end time is reached. This breaks if the system time is moved back. The attached patch breaks out of the wait if it detects the time has been moved back. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 13:36 Message: Logged In: YES user_id=341410 Originator: NO I am unsure if this is a sufficient or correct patch, but the functionality is desirable. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 From noreply at sourceforge.net Mon Apr 9 22:41:17 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 13:41:17 -0700 Subject: [Patches] [ python-Patches-1696444 ] Adding an index method to tuples Message-ID: Patches item #1696444, was opened at 2007-04-08 09:35 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Adding an index method to tuples Initial Comment: This is just a simple copy and paste from the list implementation - not exactly the most elegant solution, but perhaps a demonstration that technical limitations aren't the reason why the tuple doesn't have an index method. I suppose a common sequence function library would be best if such a feature were eventually deemed compelling. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 15:41 Message: Logged In: YES user_id=80475 Originator: NO Adding a utility function for this is silly. Don't ignore the obvious, simple solution. If a type *needs* a method, add it. If not, don't. Also, the collecitons module should be about collections, not a dumping ground for random utility functions. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 15:06 Message: Logged In: YES user_id=341410 Originator: NO I wouldn't be averse to adding utility functions to some module where it would make sense. Generally, I would say collections (it has a deque and default dictionary already), though there is already a proposed constructor called 'count' that returns the counts of all items in a sequence as a default dictionary (requiring that the items all be hashable). The reason I'm not against utility functions is because then they would be usable for 3rd party non-list, non-tuple sequences, or even usable in cases like 'a = array.array(...); i = index(a, value)'. ---------------------------------------------------------------------- Comment By: Paul Boddie (pboddie) Date: 2007-04-09 14:51 Message: Logged In: YES user_id=226443 Originator: YES See here for a recent extension of the discussion: http://groups.google.com/group/comp.lang.python/browse_frm/thread/7e9327a896c242e7/ For what it's worth, bypassing the dogma, methods like index and count on tuples are not exactly compelling in many situations (as I've pointed out before), but such issues should hardly derail the core development effort. If people consider such methods useful and if it diminishes perceived inelegance without method proliferation, just add them: people will be less frustrated and feel listened to; the language will become more regular. The only downside is any additional memory consumption. Objections related to what such methods mean, or what a tuple itself should be taken to mean, risk venturing into dogma and a consequent time-consuming discussion which takes 100 times the effort of just getting stuff done. Just look at the referenced thread (and the others of recent times) for example material. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 12:08 Message: Logged In: YES user_id=80475 Originator: NO FWIW, I am no longer satisfied with the pat answer that "tuples are not immutable lists" or the "lists for for homogenous data, tuples are for hetrogenous data". Whether tuples should grow index() and count() methods should depend solely on whether real use cases arise often enough to warrant the new methods being added. Another consideration is that this feature request and discussion comes up so often that it may be worthwhile just to give-in to conserve our limited resources of time and brain-power. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 11:24 Message: Logged In: YES user_id=341410 Originator: NO Adding an .index method to tuples suggests that a tuple is more or less an immutable list. This is not the case. Typically tuples are used to hold disparate pieces of information of different semantics and types, where as a list typically contains a list of a single type with identical meaning. For examples we merely need to look at the outputs of str.partition and str.split . Suggested close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 From noreply at sourceforge.net Mon Apr 9 22:51:03 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 13:51:03 -0700 Subject: [Patches] [ python-Patches-1690578 ] Don't block on Queue get/put when time is moved back Message-ID: Patches item #1690578, was opened at 2007-03-29 07:32 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: xiaowen (xiaowen) Assigned to: Nobody/Anonymous (nobody) Summary: Don't block on Queue get/put when time is moved back Initial Comment: When Queue get and put methods are called with a timeout, they effectively store the end time and might block until that end time is reached. This breaks if the system time is moved back. The attached patch breaks out of the wait if it detects the time has been moved back. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 15:51 Message: Logged In: YES user_id=80475 Originator: NO When a system moves time backwards, it breaks a host of invariants, not just the one in the queue module. I would not want some variant of this code used everywhere time differences are computed. Another thought is that the use case is somewhat exceptional and that the solution (readjusting endtime) isn't necessarily the right thing to do. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 15:36 Message: Logged In: YES user_id=341410 Originator: NO I am unsure if this is a sufficient or correct patch, but the functionality is desirable. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 From noreply at sourceforge.net Mon Apr 9 23:21:11 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 09 Apr 2007 14:21:11 -0700 Subject: [Patches] [ python-Patches-1690578 ] Don't block on Queue get/put when time is moved back Message-ID: Patches item #1690578, was opened at 2007-03-29 05:32 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: xiaowen (xiaowen) Assigned to: Nobody/Anonymous (nobody) Summary: Don't block on Queue get/put when time is moved back Initial Comment: When Queue get and put methods are called with a timeout, they effectively store the end time and might block until that end time is reached. This breaks if the system time is moved back. The attached patch breaks out of the wait if it detects the time has been moved back. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 14:21 Message: Logged In: YES user_id=341410 Originator: NO I agree that it would be better to fix this everywhere we can. But unless we start replacing the generic versions of everything that uses a timeout for platform-specific versions (that have guarantees regarding specified timeouts and notifications), about all we can really do is to patch each item as we go along. I suppose a good first step would be to write a generic "wait for" function that would sit in threading, _Condition.wait() is a good start. From there, one could write platform variants (Windows could use WaitForSingleObject, etc.) that don't change the end time, calling them on a platform if they exist. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 13:51 Message: Logged In: YES user_id=80475 Originator: NO When a system moves time backwards, it breaks a host of invariants, not just the one in the queue module. I would not want some variant of this code used everywhere time differences are computed. Another thought is that the use case is somewhat exceptional and that the solution (readjusting endtime) isn't necessarily the right thing to do. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 13:36 Message: Logged In: YES user_id=341410 Originator: NO I am unsure if this is a sufficient or correct patch, but the functionality is desirable. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 From noreply at sourceforge.net Tue Apr 10 18:45:14 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 10 Apr 2007 09:45:14 -0700 Subject: [Patches] [ python-Patches-925932 ] struct.pack() on 64bit architectures Message-ID: Patches item #925932, was opened at 2004-03-30 12:55 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: None >Status: Closed >Resolution: Out of Date Priority: 5 Private: No Submitted By: Aleksander Piotrowski (pelotas) Assigned to: Nobody/Anonymous (nobody) Summary: struct.pack() on 64bit architectures Initial Comment: I'm running python on OpenBSD/sparc64 (SUN Ultra 10). On this machine struct.pack() gives me: >>> struct.pack('l', 1) '\x00\x00\x00\x00\x00\x00\x00\x01' >>> struct.pack('i', 1) '\x00\x00\x00\x01' On i386 box I have: >>> struct.pack('l', 1) '\x01\x00\x00\x00' >>> struct.pack('i', 1) '\x01\x00\x00\x00' Because of this, OpenBSD port uses attached patch. I guess that this is generic problem (not OpenBSD specific) and every 64bit platform would suffer. Am I right? ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-10 18:45 Message: Logged In: YES user_id=1326842 Originator: NO test_fcntl was fixed in r38766 and r38767 to support 64bit BSD systems: http://svn.python.org/view?rev=38766&view=rev That fix is included in Python 2.4.2 and above. Thanks for the patch anyway! ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-03-31 21:08 Message: Logged In: YES user_id=21627 I don't quite understand the current code: struct flock is on Darwin struct flock { off_t l_start; /* starting offset */ off_t l_len; /* len = 0 means until end of file */ pid_t l_pid; /* lock owner */ short l_type; /* lock type: read/write, etc. */ short l_whence; /* type of l_start */ }; and off_t is typedef int64_t quad_t; typedef quad_t off_t; /* file offset */ so it appears there is no padding at all. In any case, this needs to be changed in posixfile._posixfile_.flock accordingly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-30 17:57 Message: Logged In: YES user_id=31435 For a Unix-head reviewer: the patch is to test_fcntl.py, which uses a native struct.pack() to build a lockdata argument for fcntl. On the OP's box, it constructs stuff of the wrong size, due to "l" format codes producing 8-byte thingies. The patch changes the "l" codes to "i" on the OP's platform. I wonder whether we couldn't instead use "i" codes on all BSD- ish platforms -- on 32-bit boxes, "i" is synonymous with "l", while on 64-bit boxes it looks like "l" is wrong but "i" is correct. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 From noreply at sourceforge.net Wed Apr 11 19:26:03 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 10:26:03 -0700 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 17:32 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 12:26 Message: Logged In: YES user_id=80475 Originator: NO I think we should punt on this one. When the clock changes, it is hard to detect when it happens, how much it changed, what the interactions are with other time dependent routines (logging, etc), and what to do about it. Much of this is pure guesswork. It would be different if there we some API in-place for clock changes to notify all of its clients when a change occurs. But there isn't. The responsibility for handling clock changes has to lie with the changer, not with all possible clients that call time() or sleep. Recommend closing as WontFix. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 07:55 Message: Logged In: YES user_id=21627 Originator: NO Ah. I fixed the "changing stat result problem" in Python 2.5, by not using the C library's stat implementation anymore. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-11 05:11 Message: Logged In: YES user_id=341410 Originator: NO I've never run FAT in Windows 2000. While I am pretty sure that the underlying platform doesn't have issues, Python keeps its DST offsets when it first starts up, so if you have a Python process that persists across DST changes, the modification time changes, even if the actual file modification time didn't (quitting and restarting Python "fixes" the issue). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 04:03 Message: Logged In: YES user_id=21627 Originator: NO Indeed, you would have issues with file stamps changing on Windows NT+ if the file system is FAT. However, this is not because the system time moves forth and back, but because FAT time stamps get reinterpreted according to the current time zone, not according to the time zone in force when the file was created (as NT tries to convert the FAT time stamps to UTC). On NTFS, this is not an issue, as NTFS file stamps are in UTC already (units of 100ns since Jan 1, 1601). So indeed, the submitter complains that Condition.wait doesn't time out correctly if the system clock is changed while the wait occurs. I think it is reasonable to make this complaint, however, I don't like to the patch proposed, and I do think this can't really be fixed without the help of the operating system (so I wouldn't mind if it was unfixed on systems that don't help). The point is that .wait() really takes a duration, not a time stamp, for timeout, and we should honor that. The duration should expire after the given amount of physical time has passed, independent of the system clock. As I said, one way to implement this would be to use a monotonic clock if the system has one. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 15:42 Message: Logged In: YES user_id=341410 Originator: NO Hrm. I know I've had issues with file modification times "changing" during a DST change on Windows in a Python process, but that may be due to Python only checking the timezone at startup. What you are saying sounds suspiciously like, "don't change the system clock when your program is running", which seems reasonable to me. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-10 14:26 Message: Logged In: YES user_id=21627 Originator: NO DST is irrelevant for this discussion. On WIndows NT+, the system clock (as returned by libc ftime/windows GetSystemTime) is always in UTC (broken down to a Gregorian date). It is thus not affected by DST changes. What does matter for this bug is administrator changes to the system time (like manual changes), and time adjustments due to NTP (although the Windows NTP implementation will perform a non-monototic change only during boot, AFAIK). ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 13:02 Message: Logged In: YES user_id=341410 Originator: NO Really, it would be better if time.sleep was based off of a monotonic clock, then everything could use the single working API, and we wouldn't need to monkey-patch everything that uses sleep. Unfortunately, due to the insanity that is the underlying Windows platform, the bios clock is set to local time and is converted to UTC (which is then used natively by NT). There is a work-around involving setting the system clock to UTC and setting a registry setting, but it doesn't always work, can cause 100% CPU usage during DST changes, and doesn't propagate time changes from NT to the underlying clock. http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html An alternate (or additional option) is to just disable DST changes, manually adjust times as necessary (or using a script that runs only at boot), and never run any software while adjusting the time. http://www.pcreview.co.uk/forums/thread-531048-2.php Considering the recent changes to DST, the latter link is probably more applicable. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 02:11 Message: Logged In: YES user_id=21627 Originator: NO It would be better if this was based on a monotonic clock if available on the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime). "Guessing" jumps in the time is inherently unreliable; in your code, you won't notice changes that involve less than 10s. Notice that the same problem exists for Thread.join; it would be good if they were fixed together. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Wed Apr 11 19:34:29 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 10:34:29 -0700 Subject: [Patches] [ python-Patches-1690578 ] Don't block on Queue get/put when time is moved back Message-ID: Patches item #1690578, was opened at 2007-03-29 07:32 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None >Priority: 1 Private: No Submitted By: xiaowen (xiaowen) Assigned to: Nobody/Anonymous (nobody) Summary: Don't block on Queue get/put when time is moved back Initial Comment: When Queue get and put methods are called with a timeout, they effectively store the end time and might block until that end time is reached. This breaks if the system time is moved back. The attached patch breaks out of the wait if it detects the time has been moved back. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 12:34 Message: Logged In: YES user_id=80475 Originator: NO I was recommending against "fixing" this everywhere. See related comments on pathc #1607149: Condition.wait timeout fails on clock change. Recommend closing this patch. The problem is not the responsibility of every client accessing time() or sleep(). The clients do not have enough information to reliably detect changes and to know how much the time changed. Even if those are known, it is not clear that adjusting the timeout is the always right thing to do. Also, this "problem" is somewhat obscure. Lowering the priority to 1 and recommending that the patch (and all like it) be rejected. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 16:21 Message: Logged In: YES user_id=341410 Originator: NO I agree that it would be better to fix this everywhere we can. But unless we start replacing the generic versions of everything that uses a timeout for platform-specific versions (that have guarantees regarding specified timeouts and notifications), about all we can really do is to patch each item as we go along. I suppose a good first step would be to write a generic "wait for" function that would sit in threading, _Condition.wait() is a good start. From there, one could write platform variants (Windows could use WaitForSingleObject, etc.) that don't change the end time, calling them on a platform if they exist. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 15:51 Message: Logged In: YES user_id=80475 Originator: NO When a system moves time backwards, it breaks a host of invariants, not just the one in the queue module. I would not want some variant of this code used everywhere time differences are computed. Another thought is that the use case is somewhat exceptional and that the solution (readjusting endtime) isn't necessarily the right thing to do. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 15:36 Message: Logged In: YES user_id=341410 Originator: NO I am unsure if this is a sufficient or correct patch, but the functionality is desirable. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 From noreply at sourceforge.net Wed Apr 11 19:36:59 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 10:36:59 -0700 Subject: [Patches] [ python-Patches-1685642 ] Add IllegalStateError Message-ID: Patches item #1685642, was opened at 2007-03-21 19:25 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685642&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Bj?rn Lindqvist (sonderblade) >Assigned to: Guido van Rossum (gvanrossum) Summary: Add IllegalStateError Initial Comment: Java has an exception called IllegalStateException. It is very useful for signaling "what you did would have been OK, but right now I can't do it." Python does not have a counterpart, but I think it should. For example, if you have a Python RDBMS-binding and you call db.select() before calling db.connect() it could raise a IllegalStateError (or maybe a subclass of it) that says 'IllegalStateError: "Must be connected to perform queries"'. Or for another example, take the Thread.start() function. Currently, if you call myThread.start() twice it will raise an AssertionError. That's no good because the result is undefined if you run python with -O. See Bug #904498 and Patch #1676820 (Martin v. L??wis comment). IllegalStateError should fit perfectly here: >>> t = threading.Thread() >>> t.start() >>> t.start() Traceback (most recent call last): ... IllegalStateError: thread already started ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 12:36 Message: Logged In: YES user_id=80475 Originator: NO Guido, would you pronounce on this? I think the existing framework is fine and adding a new standard exception would be unnecessary clutter. ---------------------------------------------------------------------- Comment By: Bj?rn Lindqvist (sonderblade) Date: 2007-03-26 11:29 Message: Logged In: YES user_id=51702 Originator: YES Georg, do you think that I should change the patch so that IllegalStateError sublcasses RuntimeError instead? Changing exceptions in the Standard Library to IllegalStateError would be part 2 of the patch. I don't think that should cause any trouble - only exceptions like AssertionError or base Exceptions would be changed. That's why I hope it can go into python 2.x. But bringing it up on the mailing-list is still required I guess. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 13:41 Message: Logged In: YES user_id=849994 Originator: NO In principle this is a good idea -- currently there's no guideline what to raise in such a situation. Some libraries raise RuntimeError (probably a new exception should be made subclass thereof), some do assertions, some raise custom exceptions. However, for this to be a complete patch, you'd have to find all those places where IllegalStateError would be appropriate. Also, switching to it would be a Py3k thing, provided that this breakage is suffered by python-dev. In any case, I think you should bring it up on the python-3000 mailing list -- otherwise this will never be decided upon. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685642&group_id=5470 From noreply at sourceforge.net Wed Apr 11 19:38:27 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 10:38:27 -0700 Subject: [Patches] [ python-Patches-1696393 ] Remove redundant code in ntpath.walk() Message-ID: Patches item #1696393, was opened at 2007-04-08 07:03 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696393&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Haggerty (mhagger) >Assigned to: Raymond Hettinger (rhettinger) Summary: Remove redundant code in ntpath.walk() Initial Comment: The documentation for os.listdir() explicitly says that it omits directories '.' and '..' even if they are present in the directory, and the implementation in posixmodule.c seems to agree. Yet the implementation of os.path.walk() in ntpath.py still checks for and excludes '.' and '..' from the names gotten from os.listdir(). This seems like dead code. The analogous check was removed from posixpath.py by Guido in r14534, with the comment > Remove some redundant logic from walk() -- there's no need to check > for "." and "..", since listdir() no longer returns those. The attached patch removes the extra check. It is relative to trunk r54712. I don't have a Windows computer around to test this patch, but it seems straightforward enough. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696393&group_id=5470 From noreply at sourceforge.net Wed Apr 11 19:40:49 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 10:40:49 -0700 Subject: [Patches] [ python-Patches-1677872 ] Efficient reverse line iterator Message-ID: Patches item #1677872, was opened at 2007-03-10 08:47 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1677872&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mark Russell (mark_t_russell) Assigned to: Nobody/Anonymous (nobody) Summary: Efficient reverse line iterator Initial Comment: This is an implementation of __reversed__ for the TextIOWrapper type from the new IO interface (see http://docs.google.com/Doc?id=dfksfvqd_1cn5g5m). It is used as: import io for line in reversed(io.open(filename)): ... It is efficient (only reads a block at a time) but can handle arbitrary length lines. It is useful for scanning backwards through big log files, but my main reason for submitting it is as a demonstration of the usefulness of the new IO layers - it works by putting a new buffering layer round the RawIOBase object from the open() call. It's just a proof of concept, but if there's interest in this I'm happy to write unit tests and documentation. The patch also makes io.BufferedReader support buffering, and adds a very minimal implementation of io.TextIOBase and io.TextIOWrapper (needed to make io.open() work). ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 12:40 Message: Logged In: YES user_id=80475 Originator: NO FWIW, I've wanted something like this for a long time (for scanning log files in reverse). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1677872&group_id=5470 From noreply at sourceforge.net Wed Apr 11 21:11:36 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 12:11:36 -0700 Subject: [Patches] [ python-Patches-1698723 ] Help with Python codebase Message-ID: Patches item #1698723, was opened at 2007-04-11 14:11 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698723&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Munawar (munawar2007) Assigned to: Nobody/Anonymous (nobody) Summary: Help with Python codebase Initial Comment: Hi, I am a Ph.D. student at UIUC working with Professor Ralph Johnson. My research interest is security and software architecture. At this point, I am surveying existing software architecture for buffer overflow vulnerability protection. I need some help understanding the Python codebase. In particular I have three questions. 1. Does the C code use the string library functions (strcpy, strcat, gets etc)? 2. Or does it use some sort of buffer bounds checking, either by rewriting the string library, or checking before every buffer operation? 3. Is the bounds checking available from the first release, or it has been included in a subsequent release? How did the development team go about making this change in the code? Any information would be greatly appreciated. Thanks in advance. Munawar Hafiz UIUC https://netfiles.uiuc.edu/mhafiz/www/ ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698723&group_id=5470 From noreply at sourceforge.net Wed Apr 11 21:13:59 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 12:13:59 -0700 Subject: [Patches] [ python-Patches-1698723 ] Help with Python codebase Message-ID: Patches item #1698723, was opened at 2007-04-11 19:11 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698723&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None >Status: Closed >Resolution: Invalid Priority: 5 Private: No Submitted By: Munawar (munawar2007) Assigned to: Nobody/Anonymous (nobody) Summary: Help with Python codebase Initial Comment: Hi, I am a Ph.D. student at UIUC working with Professor Ralph Johnson. My research interest is security and software architecture. At this point, I am surveying existing software architecture for buffer overflow vulnerability protection. I need some help understanding the Python codebase. In particular I have three questions. 1. Does the C code use the string library functions (strcpy, strcat, gets etc)? 2. Or does it use some sort of buffer bounds checking, either by rewriting the string library, or checking before every buffer operation? 3. Is the bounds checking available from the first release, or it has been included in a subsequent release? How did the development team go about making this change in the code? Any information would be greatly appreciated. Thanks in advance. Munawar Hafiz UIUC https://netfiles.uiuc.edu/mhafiz/www/ ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-11 19:13 Message: Logged In: YES user_id=849994 Originator: NO Please post such questions on the python-dev mailing list, to be found at . ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698723&group_id=5470 From noreply at sourceforge.net Wed Apr 11 22:40:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 13:40:33 -0700 Subject: [Patches] [ python-Patches-1698768 ] Update to Macintosh library documentation Message-ID: Patches item #1698768, was opened at 2007-04-11 15:40 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Walzer (wordtech) Assigned to: Nobody/Anonymous (nobody) Summary: Update to Macintosh library documentation Initial Comment: The attached file updates section 1 of the Macintosh library documentation, which covers getting started--installing, configuring, and learning Python on the Mac. The file updates the section to reflect changes in the Mac version of Python since 2.4 (switching to IDLE as the standard IDE, supporting universal binaries, etc.) and adds sections on application deployment with py2app, GUI toolkits, application scripting, and more. Much of the current documentation in this section is obsolete and therefore inaccurate. The updated documentation is a simple text file: I am not an expert with LaTeX. I understand the Python documentation team can handle the LaTeX formatting. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 From noreply at sourceforge.net Thu Apr 12 01:30:52 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 16:30:52 -0700 Subject: [Patches] [ python-Patches-1698768 ] Update to Macintosh library documentation Message-ID: Patches item #1698768, was opened at 2007-04-11 13:40 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Walzer (wordtech) >Assigned to: Ronald Oussoren (ronaldoussoren) Summary: Update to Macintosh library documentation Initial Comment: The attached file updates section 1 of the Macintosh library documentation, which covers getting started--installing, configuring, and learning Python on the Mac. The file updates the section to reflect changes in the Mac version of Python since 2.4 (switching to IDLE as the standard IDE, supporting universal binaries, etc.) and adds sections on application deployment with py2app, GUI toolkits, application scripting, and more. Much of the current documentation in this section is obsolete and therefore inaccurate. The updated documentation is a simple text file: I am not an expert with LaTeX. I understand the Python documentation team can handle the LaTeX formatting. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2007-04-11 16:30 Message: Logged In: YES user_id=357491 Originator: NO Thanks for the work, Kevin! Ronald, can you have a look at this and approve or reject? Once that is done we can work on converting it to LaTeX. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 From noreply at sourceforge.net Thu Apr 12 05:22:42 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 20:22:42 -0700 Subject: [Patches] [ python-Patches-1698915 ] ZipFile.printdir fix (2.5) Message-ID: Patches item #1698915, was opened at 2007-04-11 22:22 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698915&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.5) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698915&group_id=5470 From noreply at sourceforge.net Thu Apr 12 05:27:02 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 20:27:02 -0700 Subject: [Patches] [ python-Patches-1698915 ] ZipFile.printdir fix (2.5) Message-ID: Patches item #1698915, was opened at 2007-04-11 22:22 Message generated for change (Comment added) made by alanmcintyre You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698915&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.5) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). ---------------------------------------------------------------------- >Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-11 22:27 Message: Logged In: YES user_id=1115903 Originator: YES I forgot to include that this patch passes regrtest.py -r ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698915&group_id=5470 From noreply at sourceforge.net Thu Apr 12 05:27:18 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 20:27:18 -0700 Subject: [Patches] [ python-Patches-1698917 ] ZipFile.printdir fix (2.6) Message-ID: Patches item #1698917, was opened at 2007-04-11 22:27 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.6) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). Passes full regrtest.py -r ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 From noreply at sourceforge.net Thu Apr 12 08:17:44 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 11 Apr 2007 23:17:44 -0700 Subject: [Patches] [ python-Patches-1698951 ] Bastion and rexec message out-of-date Message-ID: Patches item #1698951, was opened at 2007-04-12 03:17 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698951&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Gabriel Genellina (gagenellina) Assigned to: Nobody/Anonymous (nobody) Summary: Bastion and rexec message out-of-date Initial Comment: Bastion and rexec are disabled since version 2.2. They raise a RuntimeError, but the error message is a bit outdated (it says "...in Python 2.2 and 2.3") Someone asked in c.l.p if those modules were now available again - the error message may have induced that perception, at least partially. This patch just changes the message to say "...in Python 2.2 and later" ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698951&group_id=5470 From noreply at sourceforge.net Thu Apr 12 09:01:51 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 00:01:51 -0700 Subject: [Patches] [ python-Patches-1698951 ] Bastion and rexec message out-of-date Message-ID: Patches item #1698951, was opened at 2007-04-12 06:17 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698951&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Gabriel Genellina (gagenellina) >Assigned to: Georg Brandl (gbrandl) Summary: Bastion and rexec message out-of-date Initial Comment: Bastion and rexec are disabled since version 2.2. They raise a RuntimeError, but the error message is a bit outdated (it says "...in Python 2.2 and 2.3") Someone asked in c.l.p if those modules were now available again - the error message may have induced that perception, at least partially. This patch just changes the message to say "...in Python 2.2 and later" ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-12 07:01 Message: Logged In: YES user_id=849994 Originator: NO Fixed in trunk rev. 54784. Will backport to 2.5 branch when it is unfrozen. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698951&group_id=5470 From noreply at sourceforge.net Thu Apr 12 09:58:01 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 00:58:01 -0700 Subject: [Patches] [ python-Patches-1698994 ] getstate/setstate for incremental codecs Message-ID: Patches item #1698994, was opened at 2007-04-12 09:58 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: getstate/setstate for incremental codecs Initial Comment: This patch adds getstate() and setstate() methods to incremental codecs. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 From noreply at sourceforge.net Thu Apr 12 11:37:34 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 02:37:34 -0700 Subject: [Patches] [ python-Patches-1695862 ] Fix test_urllib on Windows buildbots Message-ID: Patches item #1695862, was opened at 2007-04-07 01:03 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Ziga Seilnacht (zseil) Assigned to: Ziga Seilnacht (zseil) Summary: Fix test_urllib on Windows buildbots Initial Comment: test_urllib is currently failing on Windows buildbots because test_os at one time created a directory named test_support.TESTFN + ".2". test_os has been fixed, but the old directory is still present on buildbots. This patch is a one time fix that should be removed after the first succesful buildbot run. I added the cleanup code to test_urllib, because buildbots run tests in random order. The other option is to ask the buildbot slave maintainers to manually remove the offending directory. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-12 11:37 Message: Logged In: YES user_id=1326842 Originator: YES Commited the second patch in r54785. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-09 22:15 Message: Logged In: YES user_id=1326842 Originator: YES I checked in the first patch, and it fixed the problem on the Windows 2000 buildbot slave. I'm not entirely sure how the buildbot works, so I would like to wait with the second patch until the Windows XP slave comes back online. The test was failing only on the trunk, so there is no need to backport this change. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-09 20:10 Message: Logged In: YES user_id=33168 Originator: NO Ziga, if this passes on windows, please apply to trunk. 2.5 branch is stilly frozen (I forget if this is trunk only). Thanks for backing out my changes. They never did much. ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-07 01:06 Message: Logged In: YES user_id=1326842 Originator: YES Here is the second patch that should be applied once the first one has done its job. It also removes Neal's cleanups, because they are not needed anymore. File Added: cleanup_cleanup.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695862&group_id=5470 From noreply at sourceforge.net Thu Apr 12 12:54:07 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 03:54:07 -0700 Subject: [Patches] [ python-Patches-1698994 ] getstate/setstate for incremental codecs Message-ID: Patches item #1698994, was opened at 2007-04-12 09:58 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: getstate/setstate for incremental codecs Initial Comment: This patch adds getstate() and setstate() methods to incremental codecs. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2007-04-12 12:54 Message: Logged In: YES user_id=89016 Originator: YES This second version of the patch adds tests and fixes getstate() and setstate() for the utf-8-sig decode (and it includes the fix for the utf-8-sig decoder from r54786). File Added: diff2.txt ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 From noreply at sourceforge.net Thu Apr 12 15:55:10 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 06:55:10 -0700 Subject: [Patches] [ python-Patches-1699259 ] replacing char* with const char* in sysmodule.c/.h Message-ID: Patches item #1699259, was opened at 2007-04-12 15:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1699259&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: sebastinas (sebastinas) Assigned to: Nobody/Anonymous (nobody) Summary: replacing char* with const char* in sysmodule.c/.h Initial Comment: I'm embedding Python in a C++ application and found myself casting const char* to char* quite often when using functions like PySys_SetObject and PySys_GetObject. I attached a patch replacing the char* arguments of the above mentioned functions, PySys_GetFile and PySys_AddWarnOption with const char*. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1699259&group_id=5470 From noreply at sourceforge.net Fri Apr 13 02:59:00 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 17:59:00 -0700 Subject: [Patches] [ python-Patches-1498441 ] Change *args from a tuple to list Message-ID: Patches item #1498441, was opened at 2006-05-31 15:30 Message generated for change (Comment added) made by collinwinter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1498441&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Collin Winter (collinwinter) Assigned to: Guido van Rossum (gvanrossum) Summary: Change *args from a tuple to list Initial Comment: As discussed on python-3000, this patch changes *args from a tuple to a list. It also includes doc and test changes. The patch is against r46582. ---------------------------------------------------------------------- >Comment By: Collin Winter (collinwinter) Date: 2007-04-12 20:58 Message: Logged In: YES user_id=1344176 Originator: YES This requires too many changes in user code, eliminates optimization opportunities and generally slows things down. Rejecting. ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2006-06-01 15:32 Message: Logged In: YES user_id=1344176 As requested by Neal Norwitz, some benchmarking (all times in usecs per loop): ./python -mtimeit 'def foo(*args): pass' 'foo()' As tuple: 1.56 As list: 1.7 ./python -mtimeit 'def foo(*args): pass' 'foo(1)' As tuple: 1.75 As list: 2.04 ./python -mtimeit 'def foo(*args): pass' 'foo(1, 2)' As tuple: 1.87 As list: 2.15 ./python -mtimeit 'def foo(*args): pass' 'foo(1, 2, 3)' As tuple: 1.95 As list: 2.3 ./python -mtimeit 'def foo(*args): pass' 'foo(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)' As tuple: 2.67 As list: 2.97 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1498441&group_id=5470 From noreply at sourceforge.net Fri Apr 13 07:06:58 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 22:06:58 -0700 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 14:32 Message generated for change (Comment added) made by hashstat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- >Comment By: BC (hashstat) Date: 2007-04-12 22:06 Message: Logged In: YES user_id=939959 Originator: YES I admit that the patch isn't ideal. But it is a simple approach to fix the issue using pure Python. A better solution would be to use synchronization routines provided by the OS, such as the pthread condition functions on *NIX systems and the Win32 API synchronization functions on Windows. I wrote this patch to use with a simulator that may run for months at a time and will definitely run for days at a time. If the run happens to occur when DST ends and the clock automatically changes, as is the case for most systems, the simulation could stall for one hour and produce undesirable results. Having a sleeps depend on the wall clock is a bad idea. But since that is what we are working with, wouldn't it be better to have results skewed by a fraction of a minute or even a second rather than be off by an hour? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 10:26 Message: Logged In: YES user_id=80475 Originator: NO I think we should punt on this one. When the clock changes, it is hard to detect when it happens, how much it changed, what the interactions are with other time dependent routines (logging, etc), and what to do about it. Much of this is pure guesswork. It would be different if there we some API in-place for clock changes to notify all of its clients when a change occurs. But there isn't. The responsibility for handling clock changes has to lie with the changer, not with all possible clients that call time() or sleep. Recommend closing as WontFix. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 04:55 Message: Logged In: YES user_id=21627 Originator: NO Ah. I fixed the "changing stat result problem" in Python 2.5, by not using the C library's stat implementation anymore. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-11 02:11 Message: Logged In: YES user_id=341410 Originator: NO I've never run FAT in Windows 2000. While I am pretty sure that the underlying platform doesn't have issues, Python keeps its DST offsets when it first starts up, so if you have a Python process that persists across DST changes, the modification time changes, even if the actual file modification time didn't (quitting and restarting Python "fixes" the issue). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 01:03 Message: Logged In: YES user_id=21627 Originator: NO Indeed, you would have issues with file stamps changing on Windows NT+ if the file system is FAT. However, this is not because the system time moves forth and back, but because FAT time stamps get reinterpreted according to the current time zone, not according to the time zone in force when the file was created (as NT tries to convert the FAT time stamps to UTC). On NTFS, this is not an issue, as NTFS file stamps are in UTC already (units of 100ns since Jan 1, 1601). So indeed, the submitter complains that Condition.wait doesn't time out correctly if the system clock is changed while the wait occurs. I think it is reasonable to make this complaint, however, I don't like to the patch proposed, and I do think this can't really be fixed without the help of the operating system (so I wouldn't mind if it was unfixed on systems that don't help). The point is that .wait() really takes a duration, not a time stamp, for timeout, and we should honor that. The duration should expire after the given amount of physical time has passed, independent of the system clock. As I said, one way to implement this would be to use a monotonic clock if the system has one. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 12:42 Message: Logged In: YES user_id=341410 Originator: NO Hrm. I know I've had issues with file modification times "changing" during a DST change on Windows in a Python process, but that may be due to Python only checking the timezone at startup. What you are saying sounds suspiciously like, "don't change the system clock when your program is running", which seems reasonable to me. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-10 11:26 Message: Logged In: YES user_id=21627 Originator: NO DST is irrelevant for this discussion. On WIndows NT+, the system clock (as returned by libc ftime/windows GetSystemTime) is always in UTC (broken down to a Gregorian date). It is thus not affected by DST changes. What does matter for this bug is administrator changes to the system time (like manual changes), and time adjustments due to NTP (although the Windows NTP implementation will perform a non-monototic change only during boot, AFAIK). ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 10:02 Message: Logged In: YES user_id=341410 Originator: NO Really, it would be better if time.sleep was based off of a monotonic clock, then everything could use the single working API, and we wouldn't need to monkey-patch everything that uses sleep. Unfortunately, due to the insanity that is the underlying Windows platform, the bios clock is set to local time and is converted to UTC (which is then used natively by NT). There is a work-around involving setting the system clock to UTC and setting a registry setting, but it doesn't always work, can cause 100% CPU usage during DST changes, and doesn't propagate time changes from NT to the underlying clock. http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html An alternate (or additional option) is to just disable DST changes, manually adjust times as necessary (or using a script that runs only at boot), and never run any software while adjusting the time. http://www.pcreview.co.uk/forums/thread-531048-2.php Considering the recent changes to DST, the latter link is probably more applicable. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-02 23:11 Message: Logged In: YES user_id=21627 Originator: NO It would be better if this was based on a monotonic clock if available on the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime). "Guessing" jumps in the time is inherently unreliable; in your code, you won't notice changes that involve less than 10s. Notice that the same problem exists for Thread.join; it would be good if they were fixed together. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Fri Apr 13 08:41:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 12 Apr 2007 23:41:54 -0700 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 23:32 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-13 08:41 Message: Logged In: YES user_id=21627 Originator: NO Please let me repeat that DST changes are *not* an issue. If the system runs for many months, and has a condition timeout set that passes the DST change boundary, the condition *will* work correctly in the code as is. I don't know where you got the idea that DST changes are a problem, please rest ensured that they are no problem whatsoever, for the issue under discussion. The clock does *not* change when the time zone changes. The timeout will *not* be off by an hour. Rejecting the patch as "won't fix". ---------------------------------------------------------------------- Comment By: BC (hashstat) Date: 2007-04-13 07:06 Message: Logged In: YES user_id=939959 Originator: YES I admit that the patch isn't ideal. But it is a simple approach to fix the issue using pure Python. A better solution would be to use synchronization routines provided by the OS, such as the pthread condition functions on *NIX systems and the Win32 API synchronization functions on Windows. I wrote this patch to use with a simulator that may run for months at a time and will definitely run for days at a time. If the run happens to occur when DST ends and the clock automatically changes, as is the case for most systems, the simulation could stall for one hour and produce undesirable results. Having a sleeps depend on the wall clock is a bad idea. But since that is what we are working with, wouldn't it be better to have results skewed by a fraction of a minute or even a second rather than be off by an hour? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 19:26 Message: Logged In: YES user_id=80475 Originator: NO I think we should punt on this one. When the clock changes, it is hard to detect when it happens, how much it changed, what the interactions are with other time dependent routines (logging, etc), and what to do about it. Much of this is pure guesswork. It would be different if there we some API in-place for clock changes to notify all of its clients when a change occurs. But there isn't. The responsibility for handling clock changes has to lie with the changer, not with all possible clients that call time() or sleep. Recommend closing as WontFix. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 13:55 Message: Logged In: YES user_id=21627 Originator: NO Ah. I fixed the "changing stat result problem" in Python 2.5, by not using the C library's stat implementation anymore. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-11 11:11 Message: Logged In: YES user_id=341410 Originator: NO I've never run FAT in Windows 2000. While I am pretty sure that the underlying platform doesn't have issues, Python keeps its DST offsets when it first starts up, so if you have a Python process that persists across DST changes, the modification time changes, even if the actual file modification time didn't (quitting and restarting Python "fixes" the issue). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 10:03 Message: Logged In: YES user_id=21627 Originator: NO Indeed, you would have issues with file stamps changing on Windows NT+ if the file system is FAT. However, this is not because the system time moves forth and back, but because FAT time stamps get reinterpreted according to the current time zone, not according to the time zone in force when the file was created (as NT tries to convert the FAT time stamps to UTC). On NTFS, this is not an issue, as NTFS file stamps are in UTC already (units of 100ns since Jan 1, 1601). So indeed, the submitter complains that Condition.wait doesn't time out correctly if the system clock is changed while the wait occurs. I think it is reasonable to make this complaint, however, I don't like to the patch proposed, and I do think this can't really be fixed without the help of the operating system (so I wouldn't mind if it was unfixed on systems that don't help). The point is that .wait() really takes a duration, not a time stamp, for timeout, and we should honor that. The duration should expire after the given amount of physical time has passed, independent of the system clock. As I said, one way to implement this would be to use a monotonic clock if the system has one. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 21:42 Message: Logged In: YES user_id=341410 Originator: NO Hrm. I know I've had issues with file modification times "changing" during a DST change on Windows in a Python process, but that may be due to Python only checking the timezone at startup. What you are saying sounds suspiciously like, "don't change the system clock when your program is running", which seems reasonable to me. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-10 20:26 Message: Logged In: YES user_id=21627 Originator: NO DST is irrelevant for this discussion. On WIndows NT+, the system clock (as returned by libc ftime/windows GetSystemTime) is always in UTC (broken down to a Gregorian date). It is thus not affected by DST changes. What does matter for this bug is administrator changes to the system time (like manual changes), and time adjustments due to NTP (although the Windows NTP implementation will perform a non-monototic change only during boot, AFAIK). ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 19:02 Message: Logged In: YES user_id=341410 Originator: NO Really, it would be better if time.sleep was based off of a monotonic clock, then everything could use the single working API, and we wouldn't need to monkey-patch everything that uses sleep. Unfortunately, due to the insanity that is the underlying Windows platform, the bios clock is set to local time and is converted to UTC (which is then used natively by NT). There is a work-around involving setting the system clock to UTC and setting a registry setting, but it doesn't always work, can cause 100% CPU usage during DST changes, and doesn't propagate time changes from NT to the underlying clock. http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html An alternate (or additional option) is to just disable DST changes, manually adjust times as necessary (or using a script that runs only at boot), and never run any software while adjusting the time. http://www.pcreview.co.uk/forums/thread-531048-2.php Considering the recent changes to DST, the latter link is probably more applicable. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 08:11 Message: Logged In: YES user_id=21627 Originator: NO It would be better if this was based on a monotonic clock if available on the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime). "Guessing" jumps in the time is inherently unreliable; in your code, you won't notice changes that involve less than 10s. Notice that the same problem exists for Thread.join; it would be good if they were fixed together. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Fri Apr 13 13:44:42 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 04:44:42 -0700 Subject: [Patches] [ python-Patches-1698994 ] getstate/setstate for incremental codecs Message-ID: Patches item #1698994, was opened at 2007-04-12 09:58 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: getstate/setstate for incremental codecs Initial Comment: This patch adds getstate() and setstate() methods to incremental codecs. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2007-04-13 13:44 Message: Logged In: YES user_id=89016 Originator: YES This third version (diff3.txt) of the patch adds the special treatment of decoder state (always return (undecoded_input, additional_state_info) tuples) and adds documentation and tests. File Added: diff3.txt ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2007-04-12 12:54 Message: Logged In: YES user_id=89016 Originator: YES This second version of the patch adds tests and fixes getstate() and setstate() for the utf-8-sig decode (and it includes the fix for the utf-8-sig decoder from r54786). File Added: diff2.txt ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 From noreply at sourceforge.net Fri Apr 13 15:38:11 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 06:38:11 -0700 Subject: [Patches] [ python-Patches-1685986 ] Method cache Message-ID: Patches item #1685986, was opened at 2007-03-22 09:47 Message generated for change (Comment added) made by bioinformed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Armin Rigo (arigo) Assigned to: Raymond Hettinger (rhettinger) Summary: Method cache Initial Comment: This is a port of PyPy's method cache optimization. It gives a very good speed-up to PyPy, and it is simple enough to be attempted on CPython. The patch is against Python 2.4, which is the version I still use; I've applied the patch on the system-wide Pythons of a couple of machines a few days ago and haven't had any problem since. It gives a consistent 20+% of speed-up on all applications and benchmarks I tried, and up to 36% in extreme OO examples like the "richards" benchmark. The patch needs to be ported to the HEAD, and it needs some extra careful review. I'm a bit too busy right now to do any of these but I'll eventually come back to it if nobody else does. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 09:38 Message: Logged In: YES user_id=1768698 Originator: NO I've done some work on this patch, including porting it to HEAD, streamlining some aspects for comprehensibility, and verifying correctness. My performance results are a little less dramatic, ranging from 10% speed up for simple non-heavy OO code to about 25% for applications with deeper object hierarchies. Given the very attractive speed up, I'm happy to continue along this patch, but have a few questions for both Raymond and Armin: 1. Am I duplicating effort? 2. Should this optimization be targeted only toward the Py3k branch, as some of the trickier logic goes away (no old style classes to complicate things). 3. Should this patch be targeted to Python 2.6 as well? 4. Armin, can you explain why a type_modified call is needed in the __module__ setter? I don't immediately see how this affects the mro. 5. A minor nit: I'm not thrilled that the has version tag is cleared when the type has an unsupported base class/mro. I realize why this is convenient, but it bothers me because __bases__ is mutable and the flag cannot be reset once unset. I realize this is possibly a vanishingly rare case (__bases__ is not likely to be reset often) and that correctness is not at stake. Regardless, I'd be interested in exploring strategies that do not add more flags or complexity, but also allow for types to recover their valid version status after unsupported bases in the mro are removed. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 21:44 Message: Logged In: YES user_id=80475 Originator: NO Yes, I've got it from here. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-22 11:28 Message: Logged In: YES user_id=849994 Originator: NO Perhaps Raymond, as one of the optimization specialists, can have a look in the meantime? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 From noreply at sourceforge.net Fri Apr 13 16:32:51 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 07:32:51 -0700 Subject: [Patches] [ python-Patches-1685986 ] Method cache Message-ID: Patches item #1685986, was opened at 2007-03-22 13:47 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Armin Rigo (arigo) Assigned to: Raymond Hettinger (rhettinger) Summary: Method cache Initial Comment: This is a port of PyPy's method cache optimization. It gives a very good speed-up to PyPy, and it is simple enough to be attempted on CPython. The patch is against Python 2.4, which is the version I still use; I've applied the patch on the system-wide Pythons of a couple of machines a few days ago and haven't had any problem since. It gives a consistent 20+% of speed-up on all applications and benchmarks I tried, and up to 36% in extreme OO examples like the "richards" benchmark. The patch needs to be ported to the HEAD, and it needs some extra careful review. I'm a bit too busy right now to do any of these but I'll eventually come back to it if nobody else does. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2007-04-13 14:32 Message: Logged In: YES user_id=4771 Originator: YES About 4.: type_modified() calls are needed when either the mro or the dictionary of any type along the mro is modified. I guess that an example can be built with a class X, where X().__module__ is read twice but X.__module__ modified inbetween. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 13:38 Message: Logged In: YES user_id=1768698 Originator: NO I've done some work on this patch, including porting it to HEAD, streamlining some aspects for comprehensibility, and verifying correctness. My performance results are a little less dramatic, ranging from 10% speed up for simple non-heavy OO code to about 25% for applications with deeper object hierarchies. Given the very attractive speed up, I'm happy to continue along this patch, but have a few questions for both Raymond and Armin: 1. Am I duplicating effort? 2. Should this optimization be targeted only toward the Py3k branch, as some of the trickier logic goes away (no old style classes to complicate things). 3. Should this patch be targeted to Python 2.6 as well? 4. Armin, can you explain why a type_modified call is needed in the __module__ setter? I don't immediately see how this affects the mro. 5. A minor nit: I'm not thrilled that the has version tag is cleared when the type has an unsupported base class/mro. I realize why this is convenient, but it bothers me because __bases__ is mutable and the flag cannot be reset once unset. I realize this is possibly a vanishingly rare case (__bases__ is not likely to be reset often) and that correctness is not at stake. Regardless, I'd be interested in exploring strategies that do not add more flags or complexity, but also allow for types to recover their valid version status after unsupported bases in the mro are removed. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-03 01:44 Message: Logged In: YES user_id=80475 Originator: NO Yes, I've got it from here. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-22 15:28 Message: Logged In: YES user_id=849994 Originator: NO Perhaps Raymond, as one of the optimization specialists, can have a look in the meantime? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 From noreply at sourceforge.net Fri Apr 13 16:43:01 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 07:43:01 -0700 Subject: [Patches] [ python-Patches-1685986 ] Method cache Message-ID: Patches item #1685986, was opened at 2007-03-22 09:47 Message generated for change (Comment added) made by bioinformed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Armin Rigo (arigo) Assigned to: Raymond Hettinger (rhettinger) Summary: Method cache Initial Comment: This is a port of PyPy's method cache optimization. It gives a very good speed-up to PyPy, and it is simple enough to be attempted on CPython. The patch is against Python 2.4, which is the version I still use; I've applied the patch on the system-wide Pythons of a couple of machines a few days ago and haven't had any problem since. It gives a consistent 20+% of speed-up on all applications and benchmarks I tried, and up to 36% in extreme OO examples like the "richards" benchmark. The patch needs to be ported to the HEAD, and it needs some extra careful review. I'm a bit too busy right now to do any of these but I'll eventually come back to it if nobody else does. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 10:43 Message: Logged In: YES user_id=1768698 Originator: NO Thanks for the reminder, Armin. I've been focusing on verifying correctness for cached unbound methods, not all attributes. I'll re-review for that more general case. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2007-04-13 10:32 Message: Logged In: YES user_id=4771 Originator: YES About 4.: type_modified() calls are needed when either the mro or the dictionary of any type along the mro is modified. I guess that an example can be built with a class X, where X().__module__ is read twice but X.__module__ modified inbetween. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 09:38 Message: Logged In: YES user_id=1768698 Originator: NO I've done some work on this patch, including porting it to HEAD, streamlining some aspects for comprehensibility, and verifying correctness. My performance results are a little less dramatic, ranging from 10% speed up for simple non-heavy OO code to about 25% for applications with deeper object hierarchies. Given the very attractive speed up, I'm happy to continue along this patch, but have a few questions for both Raymond and Armin: 1. Am I duplicating effort? 2. Should this optimization be targeted only toward the Py3k branch, as some of the trickier logic goes away (no old style classes to complicate things). 3. Should this patch be targeted to Python 2.6 as well? 4. Armin, can you explain why a type_modified call is needed in the __module__ setter? I don't immediately see how this affects the mro. 5. A minor nit: I'm not thrilled that the has version tag is cleared when the type has an unsupported base class/mro. I realize why this is convenient, but it bothers me because __bases__ is mutable and the flag cannot be reset once unset. I realize this is possibly a vanishingly rare case (__bases__ is not likely to be reset often) and that correctness is not at stake. Regardless, I'd be interested in exploring strategies that do not add more flags or complexity, but also allow for types to recover their valid version status after unsupported bases in the mro are removed. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 21:44 Message: Logged In: YES user_id=80475 Originator: NO Yes, I've got it from here. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-22 11:28 Message: Logged In: YES user_id=849994 Originator: NO Perhaps Raymond, as one of the optimization specialists, can have a look in the meantime? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 From noreply at sourceforge.net Fri Apr 13 17:09:30 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 08:09:30 -0700 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 14:32 Message generated for change (Comment added) made by hashstat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Closed Resolution: Rejected Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- >Comment By: BC (hashstat) Date: 2007-04-13 08:09 Message: Logged In: YES user_id=939959 Originator: YES You are right loewis. DST does not have any effect on the wait time. I haven't looked at this in a while and I shot off a reply too quickly. The point is that a sleep (or wait) of any sort should not depend on the wall clock. While I can undertand rejecting this patch, the underlying bug still exists and I hope a solution can be provided. I plan to look into a better fix myself. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-12 23:41 Message: Logged In: YES user_id=21627 Originator: NO Please let me repeat that DST changes are *not* an issue. If the system runs for many months, and has a condition timeout set that passes the DST change boundary, the condition *will* work correctly in the code as is. I don't know where you got the idea that DST changes are a problem, please rest ensured that they are no problem whatsoever, for the issue under discussion. The clock does *not* change when the time zone changes. The timeout will *not* be off by an hour. Rejecting the patch as "won't fix". ---------------------------------------------------------------------- Comment By: BC (hashstat) Date: 2007-04-12 22:06 Message: Logged In: YES user_id=939959 Originator: YES I admit that the patch isn't ideal. But it is a simple approach to fix the issue using pure Python. A better solution would be to use synchronization routines provided by the OS, such as the pthread condition functions on *NIX systems and the Win32 API synchronization functions on Windows. I wrote this patch to use with a simulator that may run for months at a time and will definitely run for days at a time. If the run happens to occur when DST ends and the clock automatically changes, as is the case for most systems, the simulation could stall for one hour and produce undesirable results. Having a sleeps depend on the wall clock is a bad idea. But since that is what we are working with, wouldn't it be better to have results skewed by a fraction of a minute or even a second rather than be off by an hour? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 10:26 Message: Logged In: YES user_id=80475 Originator: NO I think we should punt on this one. When the clock changes, it is hard to detect when it happens, how much it changed, what the interactions are with other time dependent routines (logging, etc), and what to do about it. Much of this is pure guesswork. It would be different if there we some API in-place for clock changes to notify all of its clients when a change occurs. But there isn't. The responsibility for handling clock changes has to lie with the changer, not with all possible clients that call time() or sleep. Recommend closing as WontFix. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 04:55 Message: Logged In: YES user_id=21627 Originator: NO Ah. I fixed the "changing stat result problem" in Python 2.5, by not using the C library's stat implementation anymore. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-11 02:11 Message: Logged In: YES user_id=341410 Originator: NO I've never run FAT in Windows 2000. While I am pretty sure that the underlying platform doesn't have issues, Python keeps its DST offsets when it first starts up, so if you have a Python process that persists across DST changes, the modification time changes, even if the actual file modification time didn't (quitting and restarting Python "fixes" the issue). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 01:03 Message: Logged In: YES user_id=21627 Originator: NO Indeed, you would have issues with file stamps changing on Windows NT+ if the file system is FAT. However, this is not because the system time moves forth and back, but because FAT time stamps get reinterpreted according to the current time zone, not according to the time zone in force when the file was created (as NT tries to convert the FAT time stamps to UTC). On NTFS, this is not an issue, as NTFS file stamps are in UTC already (units of 100ns since Jan 1, 1601). So indeed, the submitter complains that Condition.wait doesn't time out correctly if the system clock is changed while the wait occurs. I think it is reasonable to make this complaint, however, I don't like to the patch proposed, and I do think this can't really be fixed without the help of the operating system (so I wouldn't mind if it was unfixed on systems that don't help). The point is that .wait() really takes a duration, not a time stamp, for timeout, and we should honor that. The duration should expire after the given amount of physical time has passed, independent of the system clock. As I said, one way to implement this would be to use a monotonic clock if the system has one. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 12:42 Message: Logged In: YES user_id=341410 Originator: NO Hrm. I know I've had issues with file modification times "changing" during a DST change on Windows in a Python process, but that may be due to Python only checking the timezone at startup. What you are saying sounds suspiciously like, "don't change the system clock when your program is running", which seems reasonable to me. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-10 11:26 Message: Logged In: YES user_id=21627 Originator: NO DST is irrelevant for this discussion. On WIndows NT+, the system clock (as returned by libc ftime/windows GetSystemTime) is always in UTC (broken down to a Gregorian date). It is thus not affected by DST changes. What does matter for this bug is administrator changes to the system time (like manual changes), and time adjustments due to NTP (although the Windows NTP implementation will perform a non-monototic change only during boot, AFAIK). ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 10:02 Message: Logged In: YES user_id=341410 Originator: NO Really, it would be better if time.sleep was based off of a monotonic clock, then everything could use the single working API, and we wouldn't need to monkey-patch everything that uses sleep. Unfortunately, due to the insanity that is the underlying Windows platform, the bios clock is set to local time and is converted to UTC (which is then used natively by NT). There is a work-around involving setting the system clock to UTC and setting a registry setting, but it doesn't always work, can cause 100% CPU usage during DST changes, and doesn't propagate time changes from NT to the underlying clock. http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html An alternate (or additional option) is to just disable DST changes, manually adjust times as necessary (or using a script that runs only at boot), and never run any software while adjusting the time. http://www.pcreview.co.uk/forums/thread-531048-2.php Considering the recent changes to DST, the latter link is probably more applicable. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-02 23:11 Message: Logged In: YES user_id=21627 Originator: NO It would be better if this was based on a monotonic clock if available on the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime). "Guessing" jumps in the time is inherently unreliable; in your code, you won't notice changes that involve less than 10s. Notice that the same problem exists for Thread.join; it would be good if they were fixed together. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Fri Apr 13 18:34:40 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 09:34:40 -0700 Subject: [Patches] [ python-Patches-1685986 ] Method cache Message-ID: Patches item #1685986, was opened at 2007-03-22 08:47 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Armin Rigo (arigo) Assigned to: Raymond Hettinger (rhettinger) Summary: Method cache Initial Comment: This is a port of PyPy's method cache optimization. It gives a very good speed-up to PyPy, and it is simple enough to be attempted on CPython. The patch is against Python 2.4, which is the version I still use; I've applied the patch on the system-wide Pythons of a couple of machines a few days ago and haven't had any problem since. It gives a consistent 20+% of speed-up on all applications and benchmarks I tried, and up to 36% in extreme OO examples like the "richards" benchmark. The patch needs to be ported to the HEAD, and it needs some extra careful review. I'm a bit too busy right now to do any of these but I'll eventually come back to it if nobody else does. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-13 11:34 Message: Logged In: YES user_id=80475 Originator: NO Also, if you've updated the patch to apply to the head, please upload it to SF so I won't have to do the same thing. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 09:43 Message: Logged In: YES user_id=1768698 Originator: NO Thanks for the reminder, Armin. I've been focusing on verifying correctness for cached unbound methods, not all attributes. I'll re-review for that more general case. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2007-04-13 09:32 Message: Logged In: YES user_id=4771 Originator: YES About 4.: type_modified() calls are needed when either the mro or the dictionary of any type along the mro is modified. I guess that an example can be built with a class X, where X().__module__ is read twice but X.__module__ modified inbetween. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 08:38 Message: Logged In: YES user_id=1768698 Originator: NO I've done some work on this patch, including porting it to HEAD, streamlining some aspects for comprehensibility, and verifying correctness. My performance results are a little less dramatic, ranging from 10% speed up for simple non-heavy OO code to about 25% for applications with deeper object hierarchies. Given the very attractive speed up, I'm happy to continue along this patch, but have a few questions for both Raymond and Armin: 1. Am I duplicating effort? 2. Should this optimization be targeted only toward the Py3k branch, as some of the trickier logic goes away (no old style classes to complicate things). 3. Should this patch be targeted to Python 2.6 as well? 4. Armin, can you explain why a type_modified call is needed in the __module__ setter? I don't immediately see how this affects the mro. 5. A minor nit: I'm not thrilled that the has version tag is cleared when the type has an unsupported base class/mro. I realize why this is convenient, but it bothers me because __bases__ is mutable and the flag cannot be reset once unset. I realize this is possibly a vanishingly rare case (__bases__ is not likely to be reset often) and that correctness is not at stake. Regardless, I'd be interested in exploring strategies that do not add more flags or complexity, but also allow for types to recover their valid version status after unsupported bases in the mro are removed. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 20:44 Message: Logged In: YES user_id=80475 Originator: NO Yes, I've got it from here. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-22 10:28 Message: Logged In: YES user_id=849994 Originator: NO Perhaps Raymond, as one of the optimization specialists, can have a look in the meantime? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 From noreply at sourceforge.net Fri Apr 13 18:41:02 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 09:41:02 -0700 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 17:32 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Closed Resolution: Rejected Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-13 11:41 Message: Logged In: YES user_id=80475 Originator: NO This needs to stay closed. It not the responsibility of the Condition code to make guesses about whether the clock has changed, by how much, and what to do about it. ---------------------------------------------------------------------- Comment By: BC (hashstat) Date: 2007-04-13 10:09 Message: Logged In: YES user_id=939959 Originator: YES You are right loewis. DST does not have any effect on the wait time. I haven't looked at this in a while and I shot off a reply too quickly. The point is that a sleep (or wait) of any sort should not depend on the wall clock. While I can undertand rejecting this patch, the underlying bug still exists and I hope a solution can be provided. I plan to look into a better fix myself. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-13 01:41 Message: Logged In: YES user_id=21627 Originator: NO Please let me repeat that DST changes are *not* an issue. If the system runs for many months, and has a condition timeout set that passes the DST change boundary, the condition *will* work correctly in the code as is. I don't know where you got the idea that DST changes are a problem, please rest ensured that they are no problem whatsoever, for the issue under discussion. The clock does *not* change when the time zone changes. The timeout will *not* be off by an hour. Rejecting the patch as "won't fix". ---------------------------------------------------------------------- Comment By: BC (hashstat) Date: 2007-04-13 00:06 Message: Logged In: YES user_id=939959 Originator: YES I admit that the patch isn't ideal. But it is a simple approach to fix the issue using pure Python. A better solution would be to use synchronization routines provided by the OS, such as the pthread condition functions on *NIX systems and the Win32 API synchronization functions on Windows. I wrote this patch to use with a simulator that may run for months at a time and will definitely run for days at a time. If the run happens to occur when DST ends and the clock automatically changes, as is the case for most systems, the simulation could stall for one hour and produce undesirable results. Having a sleeps depend on the wall clock is a bad idea. But since that is what we are working with, wouldn't it be better to have results skewed by a fraction of a minute or even a second rather than be off by an hour? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 12:26 Message: Logged In: YES user_id=80475 Originator: NO I think we should punt on this one. When the clock changes, it is hard to detect when it happens, how much it changed, what the interactions are with other time dependent routines (logging, etc), and what to do about it. Much of this is pure guesswork. It would be different if there we some API in-place for clock changes to notify all of its clients when a change occurs. But there isn't. The responsibility for handling clock changes has to lie with the changer, not with all possible clients that call time() or sleep. Recommend closing as WontFix. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 07:55 Message: Logged In: YES user_id=21627 Originator: NO Ah. I fixed the "changing stat result problem" in Python 2.5, by not using the C library's stat implementation anymore. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-11 05:11 Message: Logged In: YES user_id=341410 Originator: NO I've never run FAT in Windows 2000. While I am pretty sure that the underlying platform doesn't have issues, Python keeps its DST offsets when it first starts up, so if you have a Python process that persists across DST changes, the modification time changes, even if the actual file modification time didn't (quitting and restarting Python "fixes" the issue). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 04:03 Message: Logged In: YES user_id=21627 Originator: NO Indeed, you would have issues with file stamps changing on Windows NT+ if the file system is FAT. However, this is not because the system time moves forth and back, but because FAT time stamps get reinterpreted according to the current time zone, not according to the time zone in force when the file was created (as NT tries to convert the FAT time stamps to UTC). On NTFS, this is not an issue, as NTFS file stamps are in UTC already (units of 100ns since Jan 1, 1601). So indeed, the submitter complains that Condition.wait doesn't time out correctly if the system clock is changed while the wait occurs. I think it is reasonable to make this complaint, however, I don't like to the patch proposed, and I do think this can't really be fixed without the help of the operating system (so I wouldn't mind if it was unfixed on systems that don't help). The point is that .wait() really takes a duration, not a time stamp, for timeout, and we should honor that. The duration should expire after the given amount of physical time has passed, independent of the system clock. As I said, one way to implement this would be to use a monotonic clock if the system has one. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 15:42 Message: Logged In: YES user_id=341410 Originator: NO Hrm. I know I've had issues with file modification times "changing" during a DST change on Windows in a Python process, but that may be due to Python only checking the timezone at startup. What you are saying sounds suspiciously like, "don't change the system clock when your program is running", which seems reasonable to me. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-10 14:26 Message: Logged In: YES user_id=21627 Originator: NO DST is irrelevant for this discussion. On WIndows NT+, the system clock (as returned by libc ftime/windows GetSystemTime) is always in UTC (broken down to a Gregorian date). It is thus not affected by DST changes. What does matter for this bug is administrator changes to the system time (like manual changes), and time adjustments due to NTP (although the Windows NTP implementation will perform a non-monototic change only during boot, AFAIK). ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 13:02 Message: Logged In: YES user_id=341410 Originator: NO Really, it would be better if time.sleep was based off of a monotonic clock, then everything could use the single working API, and we wouldn't need to monkey-patch everything that uses sleep. Unfortunately, due to the insanity that is the underlying Windows platform, the bios clock is set to local time and is converted to UTC (which is then used natively by NT). There is a work-around involving setting the system clock to UTC and setting a registry setting, but it doesn't always work, can cause 100% CPU usage during DST changes, and doesn't propagate time changes from NT to the underlying clock. http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html An alternate (or additional option) is to just disable DST changes, manually adjust times as necessary (or using a script that runs only at boot), and never run any software while adjusting the time. http://www.pcreview.co.uk/forums/thread-531048-2.php Considering the recent changes to DST, the latter link is probably more applicable. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 02:11 Message: Logged In: YES user_id=21627 Originator: NO It would be better if this was based on a monotonic clock if available on the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime). "Guessing" jumps in the time is inherently unreliable; in your code, you won't notice changes that involve less than 10s. Notice that the same problem exists for Thread.join; it would be good if they were fixed together. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Fri Apr 13 21:16:08 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 12:16:08 -0700 Subject: [Patches] [ python-Patches-1700288 ] Armin's method cache optimization updated for Python 2.6 Message-ID: Patches item #1700288, was opened at 2007-04-13 15:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Jacobs (bioinformed) Assigned to: Nobody/Anonymous (nobody) Summary: Armin's method cache optimization updated for Python 2.6 Initial Comment: I've forward ported and slightly cleaned up Armin's method cache patch (see #1685986). I've attempted to clarify and tighten up several loose ends in the code, so hopefully I haven't mucked anything up. My performance results are not quite as good as Armin's, though still very encouraging. I see a typical speed up of 10%, with heavily object oriented application code seeing speedups of 15-20%. Given these rather significant results, the major task is to verify correctness. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 From noreply at sourceforge.net Fri Apr 13 21:17:07 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 12:17:07 -0700 Subject: [Patches] [ python-Patches-1685986 ] Method cache Message-ID: Patches item #1685986, was opened at 2007-03-22 09:47 Message generated for change (Comment added) made by bioinformed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Armin Rigo (arigo) Assigned to: Raymond Hettinger (rhettinger) Summary: Method cache Initial Comment: This is a port of PyPy's method cache optimization. It gives a very good speed-up to PyPy, and it is simple enough to be attempted on CPython. The patch is against Python 2.4, which is the version I still use; I've applied the patch on the system-wide Pythons of a couple of machines a few days ago and haven't had any problem since. It gives a consistent 20+% of speed-up on all applications and benchmarks I tried, and up to 36% in extreme OO examples like the "richards" benchmark. The patch needs to be ported to the HEAD, and it needs some extra careful review. I'm a bit too busy right now to do any of these but I'll eventually come back to it if nobody else does. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 15:17 Message: Logged In: YES user_id=1768698 Originator: NO Patch relative to current SVN HEAD uploaded as #1700288. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-13 12:34 Message: Logged In: YES user_id=80475 Originator: NO Also, if you've updated the patch to apply to the head, please upload it to SF so I won't have to do the same thing. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 10:43 Message: Logged In: YES user_id=1768698 Originator: NO Thanks for the reminder, Armin. I've been focusing on verifying correctness for cached unbound methods, not all attributes. I'll re-review for that more general case. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2007-04-13 10:32 Message: Logged In: YES user_id=4771 Originator: YES About 4.: type_modified() calls are needed when either the mro or the dictionary of any type along the mro is modified. I guess that an example can be built with a class X, where X().__module__ is read twice but X.__module__ modified inbetween. ---------------------------------------------------------------------- Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-13 09:38 Message: Logged In: YES user_id=1768698 Originator: NO I've done some work on this patch, including porting it to HEAD, streamlining some aspects for comprehensibility, and verifying correctness. My performance results are a little less dramatic, ranging from 10% speed up for simple non-heavy OO code to about 25% for applications with deeper object hierarchies. Given the very attractive speed up, I'm happy to continue along this patch, but have a few questions for both Raymond and Armin: 1. Am I duplicating effort? 2. Should this optimization be targeted only toward the Py3k branch, as some of the trickier logic goes away (no old style classes to complicate things). 3. Should this patch be targeted to Python 2.6 as well? 4. Armin, can you explain why a type_modified call is needed in the __module__ setter? I don't immediately see how this affects the mro. 5. A minor nit: I'm not thrilled that the has version tag is cleared when the type has an unsupported base class/mro. I realize why this is convenient, but it bothers me because __bases__ is mutable and the flag cannot be reset once unset. I realize this is possibly a vanishingly rare case (__bases__ is not likely to be reset often) and that correctness is not at stake. Regardless, I'd be interested in exploring strategies that do not add more flags or complexity, but also allow for types to recover their valid version status after unsupported bases in the mro are removed. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-02 21:44 Message: Logged In: YES user_id=80475 Originator: NO Yes, I've got it from here. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-22 11:28 Message: Logged In: YES user_id=849994 Originator: NO Perhaps Raymond, as one of the optimization specialists, can have a look in the meantime? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685986&group_id=5470 From noreply at sourceforge.net Fri Apr 13 21:22:30 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 12:22:30 -0700 Subject: [Patches] [ python-Patches-1700288 ] Armin's method cache optimization updated for Python 2.6 Message-ID: Patches item #1700288, was opened at 2007-04-13 15:16 Message generated for change (Settings changed) made by bioinformed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Jacobs (bioinformed) >Assigned to: Raymond Hettinger (rhettinger) Summary: Armin's method cache optimization updated for Python 2.6 Initial Comment: I've forward ported and slightly cleaned up Armin's method cache patch (see #1685986). I've attempted to clarify and tighten up several loose ends in the code, so hopefully I haven't mucked anything up. My performance results are not quite as good as Armin's, though still very encouraging. I see a typical speed up of 10%, with heavily object oriented application code seeing speedups of 15-20%. Given these rather significant results, the major task is to verify correctness. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 From noreply at sourceforge.net Fri Apr 13 22:38:21 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 13:38:21 -0700 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 23:32 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Closed Resolution: Rejected Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-13 22:38 Message: Logged In: YES user_id=21627 Originator: NO I agree with hashstat that this is a bug. The application specified a time-out as a duration, not as a point in time. A duration is independent of what the clock says: 10s are 10s, even if the clock gets adjusted. The proper fix would be that locks themselves get a time-out, so that Condition.wait could pass the timeout on to waiter.acquire. For thread_nt.h, the timeout could get passed to WaitForSingleObject; for thread_pthread, it could get passed to sem_timedwait (if available). Not sure about the other threading libraries. In any case, *this* issue needs to stay closed. If anybody can provide an alternative solution, please submit a new issue. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-13 18:41 Message: Logged In: YES user_id=80475 Originator: NO This needs to stay closed. It not the responsibility of the Condition code to make guesses about whether the clock has changed, by how much, and what to do about it. ---------------------------------------------------------------------- Comment By: BC (hashstat) Date: 2007-04-13 17:09 Message: Logged In: YES user_id=939959 Originator: YES You are right loewis. DST does not have any effect on the wait time. I haven't looked at this in a while and I shot off a reply too quickly. The point is that a sleep (or wait) of any sort should not depend on the wall clock. While I can undertand rejecting this patch, the underlying bug still exists and I hope a solution can be provided. I plan to look into a better fix myself. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-13 08:41 Message: Logged In: YES user_id=21627 Originator: NO Please let me repeat that DST changes are *not* an issue. If the system runs for many months, and has a condition timeout set that passes the DST change boundary, the condition *will* work correctly in the code as is. I don't know where you got the idea that DST changes are a problem, please rest ensured that they are no problem whatsoever, for the issue under discussion. The clock does *not* change when the time zone changes. The timeout will *not* be off by an hour. Rejecting the patch as "won't fix". ---------------------------------------------------------------------- Comment By: BC (hashstat) Date: 2007-04-13 07:06 Message: Logged In: YES user_id=939959 Originator: YES I admit that the patch isn't ideal. But it is a simple approach to fix the issue using pure Python. A better solution would be to use synchronization routines provided by the OS, such as the pthread condition functions on *NIX systems and the Win32 API synchronization functions on Windows. I wrote this patch to use with a simulator that may run for months at a time and will definitely run for days at a time. If the run happens to occur when DST ends and the clock automatically changes, as is the case for most systems, the simulation could stall for one hour and produce undesirable results. Having a sleeps depend on the wall clock is a bad idea. But since that is what we are working with, wouldn't it be better to have results skewed by a fraction of a minute or even a second rather than be off by an hour? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 19:26 Message: Logged In: YES user_id=80475 Originator: NO I think we should punt on this one. When the clock changes, it is hard to detect when it happens, how much it changed, what the interactions are with other time dependent routines (logging, etc), and what to do about it. Much of this is pure guesswork. It would be different if there we some API in-place for clock changes to notify all of its clients when a change occurs. But there isn't. The responsibility for handling clock changes has to lie with the changer, not with all possible clients that call time() or sleep. Recommend closing as WontFix. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 13:55 Message: Logged In: YES user_id=21627 Originator: NO Ah. I fixed the "changing stat result problem" in Python 2.5, by not using the C library's stat implementation anymore. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-11 11:11 Message: Logged In: YES user_id=341410 Originator: NO I've never run FAT in Windows 2000. While I am pretty sure that the underlying platform doesn't have issues, Python keeps its DST offsets when it first starts up, so if you have a Python process that persists across DST changes, the modification time changes, even if the actual file modification time didn't (quitting and restarting Python "fixes" the issue). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-11 10:03 Message: Logged In: YES user_id=21627 Originator: NO Indeed, you would have issues with file stamps changing on Windows NT+ if the file system is FAT. However, this is not because the system time moves forth and back, but because FAT time stamps get reinterpreted according to the current time zone, not according to the time zone in force when the file was created (as NT tries to convert the FAT time stamps to UTC). On NTFS, this is not an issue, as NTFS file stamps are in UTC already (units of 100ns since Jan 1, 1601). So indeed, the submitter complains that Condition.wait doesn't time out correctly if the system clock is changed while the wait occurs. I think it is reasonable to make this complaint, however, I don't like to the patch proposed, and I do think this can't really be fixed without the help of the operating system (so I wouldn't mind if it was unfixed on systems that don't help). The point is that .wait() really takes a duration, not a time stamp, for timeout, and we should honor that. The duration should expire after the given amount of physical time has passed, independent of the system clock. As I said, one way to implement this would be to use a monotonic clock if the system has one. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 21:42 Message: Logged In: YES user_id=341410 Originator: NO Hrm. I know I've had issues with file modification times "changing" during a DST change on Windows in a Python process, but that may be due to Python only checking the timezone at startup. What you are saying sounds suspiciously like, "don't change the system clock when your program is running", which seems reasonable to me. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-02-10 20:26 Message: Logged In: YES user_id=21627 Originator: NO DST is irrelevant for this discussion. On WIndows NT+, the system clock (as returned by libc ftime/windows GetSystemTime) is always in UTC (broken down to a Gregorian date). It is thus not affected by DST changes. What does matter for this bug is administrator changes to the system time (like manual changes), and time adjustments due to NTP (although the Windows NTP implementation will perform a non-monototic change only during boot, AFAIK). ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-02-10 19:02 Message: Logged In: YES user_id=341410 Originator: NO Really, it would be better if time.sleep was based off of a monotonic clock, then everything could use the single working API, and we wouldn't need to monkey-patch everything that uses sleep. Unfortunately, due to the insanity that is the underlying Windows platform, the bios clock is set to local time and is converted to UTC (which is then used natively by NT). There is a work-around involving setting the system clock to UTC and setting a registry setting, but it doesn't always work, can cause 100% CPU usage during DST changes, and doesn't propagate time changes from NT to the underlying clock. http://www.cl.cam.ac.uk/~mgk25/mswish/ut-rtc.html An alternate (or additional option) is to just disable DST changes, manually adjust times as necessary (or using a script that runs only at boot), and never run any software while adjusting the time. http://www.pcreview.co.uk/forums/thread-531048-2.php Considering the recent changes to DST, the latter link is probably more applicable. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 08:11 Message: Logged In: YES user_id=21627 Originator: NO It would be better if this was based on a monotonic clock if available on the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime). "Guessing" jumps in the time is inherently unreliable; in your code, you won't notice changes that involve less than 10s. Notice that the same problem exists for Thread.join; it would be good if they were fixed together. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Sat Apr 14 06:06:20 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 21:06:20 -0700 Subject: [Patches] [ python-Patches-1700463 ] VC6 build patch for trunk Message-ID: Patches item #1700463, was opened at 2007-04-14 13:06 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700463&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Nobody/Anonymous (nobody) Summary: VC6 build patch for trunk Initial Comment: Hello. This is follow up patch for current trunk. I confirmed testall.py passes on VC6 except for test_normalization.py. (I needed to expand python_d.exe's stack size to 1MB, though. otherwise, test_excepsion.py etc crashes with stack overflow) # error of test_normalization.py FAILED (errors=1) Traceback (most recent call last): File "test_normalization.py", line 94, in test_main() File "test_normalization.py", line 91, in test_main run_unittest(NormalizationTest) File "e:\python-dev\trunk\lib\test\test_support.py", line 557, in run_unittest run_suite(suite, testclass) File "e:\python-dev\trunk\lib\test\test_support.py", line 542, in run_suite raise TestFailed(err) test.test_support.TestFailed: Traceback (most recent call last): File "test_normalization.py", line 36, in test_main for line in open_urlresource(TESTDATAURL): File "e:\python-dev\trunk\lib\test\test_support.py", line 271, in open_urlreso urce requires('urlfetch') File "e:\python-dev\trunk\lib\test\test_support.py", line 94, in requires raise ResourceDenied(msg) ResourceDenied: Use of the `urlfetch' resource not enabled ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700463&group_id=5470 From noreply at sourceforge.net Sat Apr 14 06:14:36 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 21:14:36 -0700 Subject: [Patches] [ python-Patches-1700467 ] stack size of python_d.exe on VC6 Message-ID: Patches item #1700467, was opened at 2007-04-14 13:14 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700467&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Nobody/Anonymous (nobody) Summary: stack size of python_d.exe on VC6 Initial Comment: Hello. I created the patch to expand stack size of python_d.exe(debug build) on VC6 to 1MB automatically. Without this patch, test_exceptions.py (testInfiniteRecursion) causes halt with an error message dialog. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700467&group_id=5470 From noreply at sourceforge.net Sat Apr 14 08:21:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 13 Apr 2007 23:21:23 -0700 Subject: [Patches] [ python-Patches-1700288 ] Armin's method cache optimization updated for Python 2.6 Message-ID: Patches item #1700288, was opened at 2007-04-13 12:16 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Jacobs (bioinformed) Assigned to: Raymond Hettinger (rhettinger) Summary: Armin's method cache optimization updated for Python 2.6 Initial Comment: I've forward ported and slightly cleaned up Armin's method cache patch (see #1685986). I've attempted to clarify and tighten up several loose ends in the code, so hopefully I haven't mucked anything up. My performance results are not quite as good as Armin's, though still very encouraging. I see a typical speed up of 10%, with heavily object oriented application code seeing speedups of 15-20%. Given these rather significant results, the major task is to verify correctness. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-13 23:21 Message: Logged In: YES user_id=33168 Originator: NO I tried this test on parrotbench (b0.py in particular) and I'm not sure I could distinguish an improvement over the noise (best case was about 5%). The variability is pretty high on my box (dual amd64 opertons, ubuntu gcc 4.1.2-ish). At first it seemed a bit slower with the original patch which uses unsigned ints. I tried changing to unsigned longs. It seemed a little better, though not sure if it was really due to the change. I think unsigned longs should be used for 64-bit boxes. Did you use a benchmark/program that is open source? I don't know that I have anything decent to test this with. Raymond probably does though. Also what platform did you test on? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 From noreply at sourceforge.net Sat Apr 14 09:42:46 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 14 Apr 2007 00:42:46 -0700 Subject: [Patches] [ python-Patches-1696444 ] Adding an index method to tuples Message-ID: Patches item #1696444, was opened at 2007-04-08 16:35 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Boddie (pboddie) Assigned to: Nobody/Anonymous (nobody) Summary: Adding an index method to tuples Initial Comment: This is just a simple copy and paste from the list implementation - not exactly the most elegant solution, but perhaps a demonstration that technical limitations aren't the reason why the tuple doesn't have an index method. I suppose a common sequence function library would be best if such a feature were eventually deemed compelling. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-14 09:42 Message: Logged In: YES user_id=21627 Originator: NO This patch is not acceptable in its current form, because it does not include test suite and documentation changes. Furthermore, as the objective seems to be consistency across collections, I would say that the patch should either add both count and index, or none at all. Even with that added, the collections would still not behave consistently: strings have many more linear-search methods (.rindex, .find, .rfind). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 22:41 Message: Logged In: YES user_id=80475 Originator: NO Adding a utility function for this is silly. Don't ignore the obvious, simple solution. If a type *needs* a method, add it. If not, don't. Also, the collecitons module should be about collections, not a dumping ground for random utility functions. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 22:06 Message: Logged In: YES user_id=341410 Originator: NO I wouldn't be averse to adding utility functions to some module where it would make sense. Generally, I would say collections (it has a deque and default dictionary already), though there is already a proposed constructor called 'count' that returns the counts of all items in a sequence as a default dictionary (requiring that the items all be hashable). The reason I'm not against utility functions is because then they would be usable for 3rd party non-list, non-tuple sequences, or even usable in cases like 'a = array.array(...); i = index(a, value)'. ---------------------------------------------------------------------- Comment By: Paul Boddie (pboddie) Date: 2007-04-09 21:51 Message: Logged In: YES user_id=226443 Originator: YES See here for a recent extension of the discussion: http://groups.google.com/group/comp.lang.python/browse_frm/thread/7e9327a896c242e7/ For what it's worth, bypassing the dogma, methods like index and count on tuples are not exactly compelling in many situations (as I've pointed out before), but such issues should hardly derail the core development effort. If people consider such methods useful and if it diminishes perceived inelegance without method proliferation, just add them: people will be less frustrated and feel listened to; the language will become more regular. The only downside is any additional memory consumption. Objections related to what such methods mean, or what a tuple itself should be taken to mean, risk venturing into dogma and a consequent time-consuming discussion which takes 100 times the effort of just getting stuff done. Just look at the referenced thread (and the others of recent times) for example material. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 19:08 Message: Logged In: YES user_id=80475 Originator: NO FWIW, I am no longer satisfied with the pat answer that "tuples are not immutable lists" or the "lists for for homogenous data, tuples are for hetrogenous data". Whether tuples should grow index() and count() methods should depend solely on whether real use cases arise often enough to warrant the new methods being added. Another consideration is that this feature request and discussion comes up so often that it may be worthwhile just to give-in to conserve our limited resources of time and brain-power. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 18:24 Message: Logged In: YES user_id=341410 Originator: NO Adding an .index method to tuples suggests that a tuple is more or less an immutable list. This is not the case. Typically tuples are used to hold disparate pieces of information of different semantics and types, where as a list typically contains a list of a single type with identical meaning. For examples we merely need to look at the outputs of str.partition and str.split . Suggested close. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1696444&group_id=5470 From noreply at sourceforge.net Sat Apr 14 10:40:47 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 14 Apr 2007 01:40:47 -0700 Subject: [Patches] [ python-Patches-1660500 ] Hide iteration variable in list comprehensions Message-ID: Patches item #1660500, was opened at 2007-02-15 21:29 Message generated for change (Comment added) made by ncoghlan You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nick Coghlan (ncoghlan) Assigned to: Georg Brandl (gbrandl) Summary: Hide iteration variable in list comprehensions Initial Comment: This patch hides the iteration variable in list comprehensions. It adds new tests (modelled on the generator expression tests) and also removes some del statements from the standard library (where code previously cleaned up its own namespace). The changes to symtable.[ch] are more significant than strictly necessary - I found it necessary to spend some time cleaning up the code in order to understand what was needed for the list comprehension changes. Given that the 2.x and 3.0 compilers have already diverged fairly significantly, I don't believe this will make the process of keeping them in sync any more difficult than it is already. Assigning to Georg for initial review (as his set comprehensions patch provided a great deal of inspiration for this one). ---------------------------------------------------------------------- >Comment By: Nick Coghlan (ncoghlan) Date: 2007-04-14 18:40 Message: Logged In: YES user_id=1038590 Originator: YES I've uploaded a new version of the patch that is compatible with the py3k SVN branch as of April 14. The patch still includes the various cleanups I made to the symbol table processing while working out how to make this change (use sets where appropriate, avoid using the same variable name to refer to completely different things, make a couple of syntax error messages more explicit). The slowdown should be fixed at the cost of a single function call - the shorter the sequence being iterated over, the greater the percentage slowdown that will be. The speed of generator expressions should actually be (very) marginally increased as they now skip the SETUP_LOOP/POP_BLOCK steps in the same fashion as list comprehensions always have. File Added: new-comps-updated.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-03-12 22:17 Message: Logged In: YES user_id=1038590 Originator: YES The patch for 'nonlocal' was applied since Georg & I started working on this. It's going to take me a bit of fiddling to update the patch so that the parser/compiler stage play well with each other (I really should have just believed the patch utility when it reported a conflict trying to patch symtable.c). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-03-08 08:56 Message: Logged In: YES user_id=6380 Originator: NO Can't say that slowdown bothers me much, if it's typical. However I think you need to do a svn up in your workspace and regenerate the patch; I get FAILED message from patch on all the generated files and a few others: graminit.[ch], symtable.[ch], Python-ast.c, symbol.py. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-02-26 20:30 Message: Logged In: YES user_id=849994 Originator: NO Okay, I looked at the patch, and apart from a refleak in one of the compiler methods I couldn't find anything wrong. I then manually applied the rest of my set comprehension patch. The result is attached. Grammar, AST and compilation for comprehensions are now really unified. It passes the tests for listcomps, genexps and setcomps. I couldn't check properly for refleaks since e.g. test_genexps leaks in the branch head, as well as some other tests. I'm currently searching for the offending revision(s)... File Added: new-set-comps.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 22:13 Message: Logged In: YES user_id=1038590 Originator: YES For reference, the original set comprehensions patch & the py3k list discussion: http://www.python.org/sf/1548388 http://mail.python.org/pipermail/python-3000/2006-December/005188.html Note that the current patch ended up looking a *lot* like the original one (the main difference specific to list comprehensions is that the temporary list is built in the inner scope and then returned rather than being passed in as an argument. Additionally, the code has been unified only for the symtable stage - the AST and compilation stages still use separate code for listcomps and genexps). It turns out that there are some really curly scoping problems that using a nested function deals with automatically (see the new test_listcomps.py in the patch for examples). Having a more efficient mechanism specifically for 'transient' scopes like this is an interesting idea, but it's far from easy (mainly because the variables in the scope still need to be properly visible in scopes it *contains*). Adding set comprehensions on top of this patch should be pretty straightforward. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 21:53 Message: Logged In: YES user_id=1038590 Originator: YES Speed measurements show a significant speed up over trunk & Python 2.4 for module/class level code: (Python 2.4)$ python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 239 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 193 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 176 msec per loop This is almost certainly due to the variables and the list object becoming function locals. There is a slowdown inside a function (but we are still faster than Python 2.4): (Python 2.4)$ python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 259 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 176 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 185 msec per loop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 From noreply at sourceforge.net Sat Apr 14 15:49:51 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 14 Apr 2007 06:49:51 -0700 Subject: [Patches] [ python-Patches-1700288 ] Armin's method cache optimization updated for Python 2.6 Message-ID: Patches item #1700288, was opened at 2007-04-13 15:16 Message generated for change (Comment added) made by bioinformed You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kevin Jacobs (bioinformed) Assigned to: Raymond Hettinger (rhettinger) Summary: Armin's method cache optimization updated for Python 2.6 Initial Comment: I've forward ported and slightly cleaned up Armin's method cache patch (see #1685986). I've attempted to clarify and tighten up several loose ends in the code, so hopefully I haven't mucked anything up. My performance results are not quite as good as Armin's, though still very encouraging. I see a typical speed up of 10%, with heavily object oriented application code seeing speedups of 15-20%. Given these rather significant results, the major task is to verify correctness. ---------------------------------------------------------------------- >Comment By: Kevin Jacobs (bioinformed) Date: 2007-04-14 09:49 Message: Logged In: YES user_id=1768698 Originator: YES I benchmarked using pystone, pybench, and a bunch of my local scientific applications that have tight computational kernels still in pure Python. I tested on a 64bit Linux box, defining the version_tag as either int or unsigned long with no appreciable difference in performance. I'm trying to get parrotbench to run, but it doesn't seem to have been updated for modern Python versions. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-14 02:21 Message: Logged In: YES user_id=33168 Originator: NO I tried this test on parrotbench (b0.py in particular) and I'm not sure I could distinguish an improvement over the noise (best case was about 5%). The variability is pretty high on my box (dual amd64 opertons, ubuntu gcc 4.1.2-ish). At first it seemed a bit slower with the original patch which uses unsigned ints. I tried changing to unsigned longs. It seemed a little better, though not sure if it was really due to the change. I think unsigned longs should be used for 64-bit boxes. Did you use a benchmark/program that is open source? I don't know that I have anything decent to test this with. Raymond probably does though. Also what platform did you test on? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700288&group_id=5470 From noreply at sourceforge.net Sat Apr 14 16:56:56 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 14 Apr 2007 07:56:56 -0700 Subject: [Patches] [ python-Patches-1660500 ] Hide iteration variable in list comprehensions Message-ID: Patches item #1660500, was opened at 2007-02-15 06:29 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nick Coghlan (ncoghlan) Assigned to: Georg Brandl (gbrandl) Summary: Hide iteration variable in list comprehensions Initial Comment: This patch hides the iteration variable in list comprehensions. It adds new tests (modelled on the generator expression tests) and also removes some del statements from the standard library (where code previously cleaned up its own namespace). The changes to symtable.[ch] are more significant than strictly necessary - I found it necessary to spend some time cleaning up the code in order to understand what was needed for the list comprehension changes. Given that the 2.x and 3.0 compilers have already diverged fairly significantly, I don't believe this will make the process of keeping them in sync any more difficult than it is already. Assigning to Georg for initial review (as his set comprehensions patch provided a great deal of inspiration for this one). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-14 10:56 Message: Logged In: YES user_id=6380 Originator: NO Assuming all tests pass in a debug build and you don't see any leaks with "regrtest.py -R 4:3:", just check it in! I've been waiting long enough for this. Speedups are for post 3.0a1. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-04-14 04:40 Message: Logged In: YES user_id=1038590 Originator: YES I've uploaded a new version of the patch that is compatible with the py3k SVN branch as of April 14. The patch still includes the various cleanups I made to the symbol table processing while working out how to make this change (use sets where appropriate, avoid using the same variable name to refer to completely different things, make a couple of syntax error messages more explicit). The slowdown should be fixed at the cost of a single function call - the shorter the sequence being iterated over, the greater the percentage slowdown that will be. The speed of generator expressions should actually be (very) marginally increased as they now skip the SETUP_LOOP/POP_BLOCK steps in the same fashion as list comprehensions always have. File Added: new-comps-updated.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-03-12 08:17 Message: Logged In: YES user_id=1038590 Originator: YES The patch for 'nonlocal' was applied since Georg & I started working on this. It's going to take me a bit of fiddling to update the patch so that the parser/compiler stage play well with each other (I really should have just believed the patch utility when it reported a conflict trying to patch symtable.c). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-03-07 17:56 Message: Logged In: YES user_id=6380 Originator: NO Can't say that slowdown bothers me much, if it's typical. However I think you need to do a svn up in your workspace and regenerate the patch; I get FAILED message from patch on all the generated files and a few others: graminit.[ch], symtable.[ch], Python-ast.c, symbol.py. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-02-26 05:30 Message: Logged In: YES user_id=849994 Originator: NO Okay, I looked at the patch, and apart from a refleak in one of the compiler methods I couldn't find anything wrong. I then manually applied the rest of my set comprehension patch. The result is attached. Grammar, AST and compilation for comprehensions are now really unified. It passes the tests for listcomps, genexps and setcomps. I couldn't check properly for refleaks since e.g. test_genexps leaks in the branch head, as well as some other tests. I'm currently searching for the offending revision(s)... File Added: new-set-comps.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 07:13 Message: Logged In: YES user_id=1038590 Originator: YES For reference, the original set comprehensions patch & the py3k list discussion: http://www.python.org/sf/1548388 http://mail.python.org/pipermail/python-3000/2006-December/005188.html Note that the current patch ended up looking a *lot* like the original one (the main difference specific to list comprehensions is that the temporary list is built in the inner scope and then returned rather than being passed in as an argument. Additionally, the code has been unified only for the symtable stage - the AST and compilation stages still use separate code for listcomps and genexps). It turns out that there are some really curly scoping problems that using a nested function deals with automatically (see the new test_listcomps.py in the patch for examples). Having a more efficient mechanism specifically for 'transient' scopes like this is an interesting idea, but it's far from easy (mainly because the variables in the scope still need to be properly visible in scopes it *contains*). Adding set comprehensions on top of this patch should be pretty straightforward. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 06:53 Message: Logged In: YES user_id=1038590 Originator: YES Speed measurements show a significant speed up over trunk & Python 2.4 for module/class level code: (Python 2.4)$ python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 239 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 193 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 176 msec per loop This is almost certainly due to the variables and the list object becoming function locals. There is a slowdown inside a function (but we are still faster than Python 2.4): (Python 2.4)$ python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 259 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 176 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 185 msec per loop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 From noreply at sourceforge.net Sat Apr 14 19:01:09 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 14 Apr 2007 10:01:09 -0700 Subject: [Patches] [ python-Patches-1660500 ] Hide iteration variable in list comprehensions Message-ID: Patches item #1660500, was opened at 2007-02-15 11:29 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nick Coghlan (ncoghlan) Assigned to: Georg Brandl (gbrandl) Summary: Hide iteration variable in list comprehensions Initial Comment: This patch hides the iteration variable in list comprehensions. It adds new tests (modelled on the generator expression tests) and also removes some del statements from the standard library (where code previously cleaned up its own namespace). The changes to symtable.[ch] are more significant than strictly necessary - I found it necessary to spend some time cleaning up the code in order to understand what was needed for the list comprehension changes. Given that the 2.x and 3.0 compilers have already diverged fairly significantly, I don't believe this will make the process of keeping them in sync any more difficult than it is already. Assigning to Georg for initial review (as his set comprehensions patch provided a great deal of inspiration for this one). ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-14 17:01 Message: Logged In: YES user_id=849994 Originator: NO I applied this and ran regrtest -R. Found no refleaks, and the only tests that failed were compiler and transformer, as expected. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-14 14:56 Message: Logged In: YES user_id=6380 Originator: NO Assuming all tests pass in a debug build and you don't see any leaks with "regrtest.py -R 4:3:", just check it in! I've been waiting long enough for this. Speedups are for post 3.0a1. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-04-14 08:40 Message: Logged In: YES user_id=1038590 Originator: YES I've uploaded a new version of the patch that is compatible with the py3k SVN branch as of April 14. The patch still includes the various cleanups I made to the symbol table processing while working out how to make this change (use sets where appropriate, avoid using the same variable name to refer to completely different things, make a couple of syntax error messages more explicit). The slowdown should be fixed at the cost of a single function call - the shorter the sequence being iterated over, the greater the percentage slowdown that will be. The speed of generator expressions should actually be (very) marginally increased as they now skip the SETUP_LOOP/POP_BLOCK steps in the same fashion as list comprehensions always have. File Added: new-comps-updated.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-03-12 12:17 Message: Logged In: YES user_id=1038590 Originator: YES The patch for 'nonlocal' was applied since Georg & I started working on this. It's going to take me a bit of fiddling to update the patch so that the parser/compiler stage play well with each other (I really should have just believed the patch utility when it reported a conflict trying to patch symtable.c). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-03-07 22:56 Message: Logged In: YES user_id=6380 Originator: NO Can't say that slowdown bothers me much, if it's typical. However I think you need to do a svn up in your workspace and regenerate the patch; I get FAILED message from patch on all the generated files and a few others: graminit.[ch], symtable.[ch], Python-ast.c, symbol.py. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-02-26 10:30 Message: Logged In: YES user_id=849994 Originator: NO Okay, I looked at the patch, and apart from a refleak in one of the compiler methods I couldn't find anything wrong. I then manually applied the rest of my set comprehension patch. The result is attached. Grammar, AST and compilation for comprehensions are now really unified. It passes the tests for listcomps, genexps and setcomps. I couldn't check properly for refleaks since e.g. test_genexps leaks in the branch head, as well as some other tests. I'm currently searching for the offending revision(s)... File Added: new-set-comps.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 12:13 Message: Logged In: YES user_id=1038590 Originator: YES For reference, the original set comprehensions patch & the py3k list discussion: http://www.python.org/sf/1548388 http://mail.python.org/pipermail/python-3000/2006-December/005188.html Note that the current patch ended up looking a *lot* like the original one (the main difference specific to list comprehensions is that the temporary list is built in the inner scope and then returned rather than being passed in as an argument. Additionally, the code has been unified only for the symtable stage - the AST and compilation stages still use separate code for listcomps and genexps). It turns out that there are some really curly scoping problems that using a nested function deals with automatically (see the new test_listcomps.py in the patch for examples). Having a more efficient mechanism specifically for 'transient' scopes like this is an interesting idea, but it's far from easy (mainly because the variables in the scope still need to be properly visible in scopes it *contains*). Adding set comprehensions on top of this patch should be pretty straightforward. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 11:53 Message: Logged In: YES user_id=1038590 Originator: YES Speed measurements show a significant speed up over trunk & Python 2.4 for module/class level code: (Python 2.4)$ python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 239 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 193 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 176 msec per loop This is almost certainly due to the variables and the list object becoming function locals. There is a slowdown inside a function (but we are still faster than Python 2.4): (Python 2.4)$ python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 259 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 176 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 185 msec per loop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 From noreply at sourceforge.net Sat Apr 14 19:42:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 14 Apr 2007 10:42:33 -0700 Subject: [Patches] [ python-Patches-1660500 ] Hide iteration variable in list comprehensions Message-ID: Patches item #1660500, was opened at 2007-02-15 06:29 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nick Coghlan (ncoghlan) Assigned to: Georg Brandl (gbrandl) Summary: Hide iteration variable in list comprehensions Initial Comment: This patch hides the iteration variable in list comprehensions. It adds new tests (modelled on the generator expression tests) and also removes some del statements from the standard library (where code previously cleaned up its own namespace). The changes to symtable.[ch] are more significant than strictly necessary - I found it necessary to spend some time cleaning up the code in order to understand what was needed for the list comprehension changes. Given that the 2.x and 3.0 compilers have already diverged fairly significantly, I don't believe this will make the process of keeping them in sync any more difficult than it is already. Assigning to Georg for initial review (as his set comprehensions patch provided a great deal of inspiration for this one). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-14 13:42 Message: Logged In: YES user_id=6380 Originator: NO OK, I think Nick can check it in himself, right Nick? ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-04-14 13:01 Message: Logged In: YES user_id=849994 Originator: NO I applied this and ran regrtest -R. Found no refleaks, and the only tests that failed were compiler and transformer, as expected. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-14 10:56 Message: Logged In: YES user_id=6380 Originator: NO Assuming all tests pass in a debug build and you don't see any leaks with "regrtest.py -R 4:3:", just check it in! I've been waiting long enough for this. Speedups are for post 3.0a1. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-04-14 04:40 Message: Logged In: YES user_id=1038590 Originator: YES I've uploaded a new version of the patch that is compatible with the py3k SVN branch as of April 14. The patch still includes the various cleanups I made to the symbol table processing while working out how to make this change (use sets where appropriate, avoid using the same variable name to refer to completely different things, make a couple of syntax error messages more explicit). The slowdown should be fixed at the cost of a single function call - the shorter the sequence being iterated over, the greater the percentage slowdown that will be. The speed of generator expressions should actually be (very) marginally increased as they now skip the SETUP_LOOP/POP_BLOCK steps in the same fashion as list comprehensions always have. File Added: new-comps-updated.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-03-12 08:17 Message: Logged In: YES user_id=1038590 Originator: YES The patch for 'nonlocal' was applied since Georg & I started working on this. It's going to take me a bit of fiddling to update the patch so that the parser/compiler stage play well with each other (I really should have just believed the patch utility when it reported a conflict trying to patch symtable.c). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-03-07 17:56 Message: Logged In: YES user_id=6380 Originator: NO Can't say that slowdown bothers me much, if it's typical. However I think you need to do a svn up in your workspace and regenerate the patch; I get FAILED message from patch on all the generated files and a few others: graminit.[ch], symtable.[ch], Python-ast.c, symbol.py. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-02-26 05:30 Message: Logged In: YES user_id=849994 Originator: NO Okay, I looked at the patch, and apart from a refleak in one of the compiler methods I couldn't find anything wrong. I then manually applied the rest of my set comprehension patch. The result is attached. Grammar, AST and compilation for comprehensions are now really unified. It passes the tests for listcomps, genexps and setcomps. I couldn't check properly for refleaks since e.g. test_genexps leaks in the branch head, as well as some other tests. I'm currently searching for the offending revision(s)... File Added: new-set-comps.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 07:13 Message: Logged In: YES user_id=1038590 Originator: YES For reference, the original set comprehensions patch & the py3k list discussion: http://www.python.org/sf/1548388 http://mail.python.org/pipermail/python-3000/2006-December/005188.html Note that the current patch ended up looking a *lot* like the original one (the main difference specific to list comprehensions is that the temporary list is built in the inner scope and then returned rather than being passed in as an argument. Additionally, the code has been unified only for the symtable stage - the AST and compilation stages still use separate code for listcomps and genexps). It turns out that there are some really curly scoping problems that using a nested function deals with automatically (see the new test_listcomps.py in the patch for examples). Having a more efficient mechanism specifically for 'transient' scopes like this is an interesting idea, but it's far from easy (mainly because the variables in the scope still need to be properly visible in scopes it *contains*). Adding set comprehensions on top of this patch should be pretty straightforward. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 06:53 Message: Logged In: YES user_id=1038590 Originator: YES Speed measurements show a significant speed up over trunk & Python 2.4 for module/class level code: (Python 2.4)$ python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 239 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 193 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 176 msec per loop This is almost certainly due to the variables and the list object becoming function locals. There is a slowdown inside a function (but we are still faster than Python 2.4): (Python 2.4)$ python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 259 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 176 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 185 msec per loop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 From noreply at sourceforge.net Sat Apr 14 23:27:19 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 14 Apr 2007 14:27:19 -0700 Subject: [Patches] [ python-Patches-1667860 ] urllib2 raises an UnboundLocalError if "auth-int" is the qop Message-ID: Patches item #1667860, was opened at 2007-02-24 15:19 Message generated for change (Comment added) made by varmaa You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1667860&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Atul Varma (varmaa) Assigned to: Georg Brandl (gbrandl) Summary: urllib2 raises an UnboundLocalError if "auth-int" is the qop Initial Comment: If a proxy server is connected to that specifies the "auth-int" quality of protection (qop) code--or any qop code aside from "auth", actually--urllib2 raises an UnboundLocalError exception. While this patch doesn't implement auth-int, it does modify the behavior of urllib2 such that it raises a URLError with the reason "qop '%s' is not supported', where %s is the name of the qop code. Two unit tests that test urllib2's functionality with an in-process proxy can be found at: http://varmaa.googlepages.com/Urllib2ProxyTests.py I will try to attach this file to this patch ticket if I can. I am also interested in integrating this unit testing suite with the Python standard lib unit tests, and am willing to work with you if this is desirable. One of the unit tests in this suite succeeds with the standard version of urllib2.py, but the other fails unless this patch is applied. This patch (and the unit test suite) applies to Python 2.4 and 2.5. ---------------------------------------------------------------------- >Comment By: Atul Varma (varmaa) Date: 2007-04-14 21:27 Message: Logged In: YES user_id=863202 Originator: YES Sorry for the delay; I've added two failure cases to the testing suite--one which supplies bad password information and another that has no password information to supply--so the patch should now be complete. Please let me know if you need anything else. File Added: test_urllib2_localnet.py ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-13 08:08 Message: Logged In: YES user_id=849994 Originator: NO The new test file looks good, so if you complete the patch I'll apply it. ---------------------------------------------------------------------- Comment By: Atul Varma (varmaa) Date: 2007-02-26 01:42 Message: Logged In: YES user_id=863202 Originator: YES Thanks for the feedback, John--I've applied your suggestions to the testing suite (including renaming the file name) and the patch file. The only thing I haven't done yet is add tests for failure cases, but I wanted to submit the "refactored" version of the test suite before doing that so I know I'm on the right track. ---------------------------------------------------------------------- Comment By: Atul Varma (varmaa) Date: 2007-02-26 01:38 Message: Logged In: YES user_id=863202 Originator: YES File Added: test_urllib2_localnet.py ---------------------------------------------------------------------- Comment By: Atul Varma (varmaa) Date: 2007-02-26 01:33 Message: Logged In: YES user_id=863202 Originator: YES File Added: urllib2_patch.diff ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2007-02-25 17:37 Message: Logged In: YES user_id=261020 Originator: NO This looks useful. It would be nice to test failure cases too (e.g. wrong password, .add_password() not called, ...). I think if this goes into a new file (which might be reasonable), it should be named something like test_urllib2_localnet.py , to allow for adding more loopback tests, which will not always involve proxy handling. I suppose it should NOT depend on the network resource flag (Lib/test/regrtest.py -u network) since it's only localhost. Style (see PEP 8): Don't put whitespace inside brackets -- e.g. bad: fn( arg ) good: fn(arg) Use this_kind_of_name or thiskind if short, not thisKind. Imports and classes etc. don't need comments to label them as such. Probably is useful to label the BaseHTTPServer code though (maybe '# Local proxy test infrastructure'). See e.g. end of Lib/test/test_urllib2net.py for appropriate stdlib test boilerplate. Your patch has inconsistent line endings, BTW. ---------------------------------------------------------------------- Comment By: Atul Varma (varmaa) Date: 2007-02-24 15:20 Message: Logged In: YES user_id=863202 Originator: YES File Added: Urllib2ProxyTests.py ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1667860&group_id=5470 From noreply at sourceforge.net Sun Apr 15 09:28:05 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 15 Apr 2007 00:28:05 -0700 Subject: [Patches] [ python-Patches-1690578 ] Don't block on Queue get/put when time is moved back Message-ID: Patches item #1690578, was opened at 2007-03-29 14:32 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Rejected Priority: 1 Private: No Submitted By: xiaowen (xiaowen) Assigned to: Nobody/Anonymous (nobody) Summary: Don't block on Queue get/put when time is moved back Initial Comment: When Queue get and put methods are called with a timeout, they effectively store the end time and might block until that end time is reached. This breaks if the system time is moved back. The attached patch breaks out of the wait if it detects the time has been moved back. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-15 09:28 Message: Logged In: YES user_id=21627 Originator: NO Rejecting this patch for the same reason #1607149 was rejected. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 19:34 Message: Logged In: YES user_id=80475 Originator: NO I was recommending against "fixing" this everywhere. See related comments on pathc #1607149: Condition.wait timeout fails on clock change. Recommend closing this patch. The problem is not the responsibility of every client accessing time() or sleep(). The clients do not have enough information to reliably detect changes and to know how much the time changed. Even if those are known, it is not clear that adjusting the timeout is the always right thing to do. Also, this "problem" is somewhat obscure. Lowering the priority to 1 and recommending that the patch (and all like it) be rejected. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 23:21 Message: Logged In: YES user_id=341410 Originator: NO I agree that it would be better to fix this everywhere we can. But unless we start replacing the generic versions of everything that uses a timeout for platform-specific versions (that have guarantees regarding specified timeouts and notifications), about all we can really do is to patch each item as we go along. I suppose a good first step would be to write a generic "wait for" function that would sit in threading, _Condition.wait() is a good start. From there, one could write platform variants (Windows could use WaitForSingleObject, etc.) that don't change the end time, calling them on a platform if they exist. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-09 22:51 Message: Logged In: YES user_id=80475 Originator: NO When a system moves time backwards, it breaks a host of invariants, not just the one in the queue module. I would not want some variant of this code used everywhere time differences are computed. Another thought is that the use case is somewhat exceptional and that the solution (readjusting endtime) isn't necessarily the right thing to do. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2007-04-09 22:36 Message: Logged In: YES user_id=341410 Originator: NO I am unsure if this is a sufficient or correct patch, but the functionality is desirable. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690578&group_id=5470 From noreply at sourceforge.net Sun Apr 15 12:12:32 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 15 Apr 2007 03:12:32 -0700 Subject: [Patches] [ python-Patches-1698994 ] getstate/setstate for incremental codecs Message-ID: Patches item #1698994, was opened at 2007-04-12 09:58 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: getstate/setstate for incremental codecs Initial Comment: This patch adds getstate() and setstate() methods to incremental codecs. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2007-04-15 12:12 Message: Logged In: YES user_id=89016 Originator: YES This fourth version of the patch (diff4.txt) changes the specification, so that IncrementalEncoder.getstate() and IncrementalDecoder.getstate()[1] must be integers. File Added: diff4.txt ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2007-04-13 13:44 Message: Logged In: YES user_id=89016 Originator: YES This third version (diff3.txt) of the patch adds the special treatment of decoder state (always return (undecoded_input, additional_state_info) tuples) and adds documentation and tests. File Added: diff3.txt ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2007-04-12 12:54 Message: Logged In: YES user_id=89016 Originator: YES This second version of the patch adds tests and fixes getstate() and setstate() for the utf-8-sig decode (and it includes the fix for the utf-8-sig decoder from r54786). File Added: diff2.txt ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 From noreply at sourceforge.net Sun Apr 15 14:06:52 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 15 Apr 2007 05:06:52 -0700 Subject: [Patches] [ python-Patches-1660500 ] Hide iteration variable in list comprehensions Message-ID: Patches item #1660500, was opened at 2007-02-15 21:29 Message generated for change (Comment added) made by ncoghlan You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nick Coghlan (ncoghlan) Assigned to: Georg Brandl (gbrandl) Summary: Hide iteration variable in list comprehensions Initial Comment: This patch hides the iteration variable in list comprehensions. It adds new tests (modelled on the generator expression tests) and also removes some del statements from the standard library (where code previously cleaned up its own namespace). The changes to symtable.[ch] are more significant than strictly necessary - I found it necessary to spend some time cleaning up the code in order to understand what was needed for the list comprehension changes. Given that the 2.x and 3.0 compilers have already diverged fairly significantly, I don't believe this will make the process of keeping them in sync any more difficult than it is already. Assigning to Georg for initial review (as his set comprehensions patch provided a great deal of inspiration for this one). ---------------------------------------------------------------------- >Comment By: Nick Coghlan (ncoghlan) Date: 2007-04-15 22:06 Message: Logged In: YES user_id=1038590 Originator: YES test_compiler and test_transformer are failing due to the fact that the compiler package still needs some TLC to catch up with the Grammar changes. test_logging, test_tcl and test_structmembers all fail when run with -R 4:3:. However, these failures don't appear to be due to this change. More importantly, no leaks are reported in the new tests that are part of this update (test_listcomps, test_setcomps). Committed as rev 54835. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-15 03:42 Message: Logged In: YES user_id=6380 Originator: NO OK, I think Nick can check it in himself, right Nick? ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-04-15 03:01 Message: Logged In: YES user_id=849994 Originator: NO I applied this and ran regrtest -R. Found no refleaks, and the only tests that failed were compiler and transformer, as expected. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-15 00:56 Message: Logged In: YES user_id=6380 Originator: NO Assuming all tests pass in a debug build and you don't see any leaks with "regrtest.py -R 4:3:", just check it in! I've been waiting long enough for this. Speedups are for post 3.0a1. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-04-14 18:40 Message: Logged In: YES user_id=1038590 Originator: YES I've uploaded a new version of the patch that is compatible with the py3k SVN branch as of April 14. The patch still includes the various cleanups I made to the symbol table processing while working out how to make this change (use sets where appropriate, avoid using the same variable name to refer to completely different things, make a couple of syntax error messages more explicit). The slowdown should be fixed at the cost of a single function call - the shorter the sequence being iterated over, the greater the percentage slowdown that will be. The speed of generator expressions should actually be (very) marginally increased as they now skip the SETUP_LOOP/POP_BLOCK steps in the same fashion as list comprehensions always have. File Added: new-comps-updated.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-03-12 22:17 Message: Logged In: YES user_id=1038590 Originator: YES The patch for 'nonlocal' was applied since Georg & I started working on this. It's going to take me a bit of fiddling to update the patch so that the parser/compiler stage play well with each other (I really should have just believed the patch utility when it reported a conflict trying to patch symtable.c). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-03-08 08:56 Message: Logged In: YES user_id=6380 Originator: NO Can't say that slowdown bothers me much, if it's typical. However I think you need to do a svn up in your workspace and regenerate the patch; I get FAILED message from patch on all the generated files and a few others: graminit.[ch], symtable.[ch], Python-ast.c, symbol.py. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-02-26 20:30 Message: Logged In: YES user_id=849994 Originator: NO Okay, I looked at the patch, and apart from a refleak in one of the compiler methods I couldn't find anything wrong. I then manually applied the rest of my set comprehension patch. The result is attached. Grammar, AST and compilation for comprehensions are now really unified. It passes the tests for listcomps, genexps and setcomps. I couldn't check properly for refleaks since e.g. test_genexps leaks in the branch head, as well as some other tests. I'm currently searching for the offending revision(s)... File Added: new-set-comps.diff ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 22:13 Message: Logged In: YES user_id=1038590 Originator: YES For reference, the original set comprehensions patch & the py3k list discussion: http://www.python.org/sf/1548388 http://mail.python.org/pipermail/python-3000/2006-December/005188.html Note that the current patch ended up looking a *lot* like the original one (the main difference specific to list comprehensions is that the temporary list is built in the inner scope and then returned rather than being passed in as an argument. Additionally, the code has been unified only for the symtable stage - the AST and compilation stages still use separate code for listcomps and genexps). It turns out that there are some really curly scoping problems that using a nested function deals with automatically (see the new test_listcomps.py in the patch for examples). Having a more efficient mechanism specifically for 'transient' scopes like this is an interesting idea, but it's far from easy (mainly because the variables in the scope still need to be properly visible in scopes it *contains*). Adding set comprehensions on top of this patch should be pretty straightforward. ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2007-02-15 21:53 Message: Logged In: YES user_id=1038590 Originator: YES Speed measurements show a significant speed up over trunk & Python 2.4 for module/class level code: (Python 2.4)$ python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 239 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 193 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" "[[x for x in seq] for y in seq]" 10 loops, best of 3: 176 msec per loop This is almost certainly due to the variables and the list object becoming function locals. There is a slowdown inside a function (but we are still faster than Python 2.4): (Python 2.4)$ python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 259 msec per loop (Python 2.x trunk)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 176 msec per loop (Python 3000)$ ./python -m timeit -s "seq=range(1000)" -s "def f(): return [[x for x in seq] for y in seq]" "f()" 10 loops, best of 3: 185 msec per loop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1660500&group_id=5470 From noreply at sourceforge.net Sun Apr 15 21:46:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 15 Apr 2007 12:46:23 -0700 Subject: [Patches] [ python-Patches-850482 ] Enhance frame handing in warnings.warn() Message-ID: Patches item #850482, was opened at 2003-11-28 00:12 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=850482&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Enhance frame handing in warnings.warn() Initial Comment: This patch enhances warnings.warn() in the following way: The stacklevel passed in may be negative. In that case the call stack is searched for the innermost frame whose module name differs in the first -stacklevel components. This frame will be used in the report. So when you have the following call stack: m1.f() m1.m11.f() m1.m12.f() m2.m21.f() m2.m22.f() and the innermost function() m2.m22.f() call warnings.warn() with a stacklevel of -2 the frame reported will be from m1.m12.f(), because it is the first one from outside the m2 package. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-15 21:46 Message: Logged In: YES user_id=1326842 Originator: NO Looks good to me. I like that skipmodules is a regular expression, but I don't know how will it affect the intended rewrite in C: http://www.python.org/sf/1631171 It would be nice if the documentation mentioned what is the interaction between stacklevel and skipmodules parameters, at least something like: "If you want to issue a warning outside your module, and you don't know how many frames will have to be skipped for that, pass some big integer for stacklevel" If this patch is accepted, it should be put to use in the _struct module: >>> struct.Struct(" Patches item #1692664, was opened at 2007-04-01 23:23 Message generated for change (Comment added) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-15 19:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 15:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 01:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Tue Apr 17 00:15:24 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 16 Apr 2007 15:15:24 -0700 Subject: [Patches] [ python-Patches-1698994 ] getstate/setstate for incremental codecs Message-ID: Patches item #1698994, was opened at 2007-04-12 09:58 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Guido van Rossum (gvanrossum) Summary: getstate/setstate for incremental codecs Initial Comment: This patch adds getstate() and setstate() methods to incremental codecs. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2007-04-17 00:15 Message: Logged In: YES user_id=89016 Originator: YES Checked in as r54845. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2007-04-15 12:12 Message: Logged In: YES user_id=89016 Originator: YES This fourth version of the patch (diff4.txt) changes the specification, so that IncrementalEncoder.getstate() and IncrementalDecoder.getstate()[1] must be integers. File Added: diff4.txt ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2007-04-13 13:44 Message: Logged In: YES user_id=89016 Originator: YES This third version (diff3.txt) of the patch adds the special treatment of decoder state (always return (undecoded_input, additional_state_info) tuples) and adds documentation and tests. File Added: diff3.txt ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2007-04-12 12:54 Message: Logged In: YES user_id=89016 Originator: YES This second version of the patch adds tests and fixes getstate() and setstate() for the utf-8-sig decode (and it includes the fix for the utf-8-sig decoder from r54786). File Added: diff2.txt ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698994&group_id=5470 From noreply at sourceforge.net Tue Apr 17 06:06:01 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 16 Apr 2007 21:06:01 -0700 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 05:08 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Closed Resolution: Fixed Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Raymond Hettinger (rhettinger) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-16 21:06 Message: Logged In: YES user_id=33168 Originator: NO This was reverted on the 2.5 branch due to the change in behavior. See http://mail.python.org/pipermail/python-dev/2007-April/072565.html for more details. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-02-07 12:17 Message: Logged In: YES user_id=80475 Originator: NO Added PyDict_CheckExact() in revisions 53655 and 53656. A side-effect of this change is to slow-down updates with dict subclasses that are not overriding keys() and __getitem__(). This is especially unfortunate given good existing alternatives and given a lack of real use cases (dict subclasses that aspire to hand-off updates but not use their actual keys and mapped values). Left out the gratuitous expansion of the API which exposed too much of the internal implementation and sought to introduce more implicit behaviors that would better be handled by explictly passing in an iterable of items to d.update(). For example. d.update((k(x), g(x)) for x in myweirdmapping). ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-20 04:59 Message: Logged In: YES user_id=89016 Originator: YES To clear up some apparent misunderstandings: This patch does *not* advocate that some dict methods should be implemented by calling other dict methods so that dict subclasses only have to overwrite a few methods to gain a completely consistent implementation. This patch only fixes the dict constructor (and update()) and consists of two parts: (1) There are three code paths in dictobject.c::dict_update_common(): (a) if the constructor argument doesn't have a "keys" attribute treat it as a iterable of items; (b) if the argument has a "keys" attribute, but is not a dict (and not an instance of a subclass of dict), use keys() and __getitem__() to make a copy of the mapping-like object. (c) if the argument has a "keys" attribute and is a dict (or an instance of a subclass of dict) bypass any of the overwritten methods that the object might provide and directly use the dict implementation. This patch changes PyDict_Merge() so that code path (b) is used for dict constructor arguments that are subclasses of dict, so that any overwritten methods are honored. (2) This means that now if a subclass of dict is passed to the constructor or update() the code is IMHO more correct (it honors the reimplemenation of the mapping methods), but slower. To reduce the slowdown instead of using kesY() and __getitem__(), iterkeys() and __getitem__() are used. I can't see why the current behaviour should be better: Yes, it is faster, but it is also wrong: Without the patch the behaviour of dict() and dict.update() depends on the fact whether the argument happens to subclass dict or not. If it doesn't all is well: the argument is treated as a mapping (i.e. keys() and __getitem__() are used) otherwise the methods are completely ignored. So can we agree on the fact that (1) is desirable? (At least Guido said as much: http://mail.python.org/pipermail/python-dev/2006-December/070341.html) BTW, I only added PyMapping_Iterkeys() because it mirrors PyMapping_Keys(). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 16:13 Message: Logged In: YES user_id=80475 Originator: NO Since update already supports (key, item) changes, I do not see that rationale in trying to expand the definition what is dict-like to include a try-this, then try-that approach. This is a little too ad-hoc for too little benefit. Also, I do not see the point of adding PyMapping_Iterkeys to the C API. It affords no advantage over its macro definition (the current one-way-to-do-it). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 15:00 Message: Logged In: YES user_id=80475 Originator: NO It is also asking for bugs if someone hooks __getitem__ and starts to make possibly invalid assumptions about what other changes occur implicitly. Also, this patch kills the performance of builtin subclasses. If I subclass dict to add a new method, it would suck to have the performance of all of the other methods drop percariously. This patch is somewhat overzealous. It encroaches on the terriority of UserDict.DictMixin which was specifically made for propagating new behaviors. It unnecessarily exposes implementation details. It introduces implicit behaviors that should be done through explicit overrides of methods whose behavior is supposed to change. And, it is on the top of a slippery slope that we don't want to go down (i.e. do we want to guarantee that list.append is implemented in terms of list.extend, etc). Python has no shortage of places where builtin subclasses make direct calls to the underlying C code -- this patch leads to a bottomless pit of changes that kill performance and make implicit side-effects the norm instead of the exception. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 14:29 Message: Logged In: YES user_id=764593 Originator: NO FWIW, I'm not sure I agree on not specifying which methods call share implementation. If someone hooks __getitem__ but not get, that is just asking for bugs. (The implementation of get may -- but need not -- make its own call to __getitem__, and not everyone will make the same decision.) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 14:26 Message: Logged In: YES user_id=764593 Originator: NO As I understand it, the problem is that dict.update is assuming any dict subclass will use the same internal data representation. Restricting the fast path to exactly builtin dicts (not subclasses) fixes the bug, but makes the fallback more frequent. The existing fallback is to call keys(), then iterate over it, retrieving the value for each key. (keys is required for a "minimal mapping" as documented is UserDict, and a few other random places.) The only potential dependency on other methods is his proposed new intermediate path that avoids creating a list of all keys, by using iterkeys if it exists. (I suggested using iteritems to avoid the lookups.) If iter* aren't implemented, the only harm is falling back to the existing fallback of "for k in keys():" ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 13:07 Message: Logged In: YES user_id=80475 Originator: NO I'm -1 on making ANY guarantees about which methods underlie others -- that would constitute new and everlasting guarantees about how mappings are implemented. Subclasses should explicity override/extend the methods withed changed behavior. If that proves non-trivial, then it is likely there should be a has-a relationship instead of an is-a relationship. Also, it is likely that the subclass will have Liskov substitutability violations. Either way, there is probably a design flaw. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 11:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 09:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Tue Apr 17 12:38:25 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 17 Apr 2007 03:38:25 -0700 Subject: [Patches] [ python-Patches-1552880 ] Unicode Imports Message-ID: Patches item #1552880, was opened at 2006-09-05 18:11 Message generated for change (Comment added) made by krisvale You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1552880&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kristj?n Valur (krisvale) Assigned to: Nobody/Anonymous (nobody) Summary: Unicode Imports Initial Comment: This patch modifies the import mechanism to fully support unicode pathnames on Windows. It does this by first converting each member of sys.path to utf-8. strings are encoded using the current locale. The whole of the import logic is then unchanged and works on the utf-8 strings as though they were regular ascii strings in the current locale. Only when file operations are done, such as stat() and open(), do we then convert from utf-8 back to unicode and use the Windows unicode APIs for the job. This is also done when initializing Module objects. This approach has the benefit of being of having a low impact on the importing logic, and is thus easy to verify. There is however some overhead with the conversions. At CCP games we used this approach, backported to python 2.3, to get unicode imports working for our game, EVE Online, and thereby solving installation issues in the far east. This patch is submitted as demonstration code to the python community. I would like to see unicode fully supported in 2.6. Cheers, Kristj??n ---------------------------------------------------------------------- >Comment By: Kristj?n Valur (krisvale) Date: 2007-04-17 10:38 Message: Logged In: YES user_id=1262199 Originator: YES I have uplodaed unicodepatch4.patch, which simplifies this a bit. __file__ and __path__ components are now stored in filesystemencoding if possible. and non-unicode paths are assumed to be in filesystemencoding. This minimizes the impact of the change. File Added: Unicodeimport4.patch ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-09-12 20:17 Message: Logged In: YES user_id=21627 krisvale: indeed, option 4 is platform dependent. Notice that on Linux, the file system encoding won't necessarily be UTF-8. Instead, the value depends on the locale, so it may be latin-1, latin-9, gb2312, ... This makes it even more dependent on the platform, and even the current user being logged in (such is life with locale-based approaches; the same is mostly true for Windows: "mbcs" can mean nearly anything). option 1) is Py3k-safe, where path names will be Unicode strings always. As you say, Unicode is a virulent type, so this approach would need a wide consensus. I'm personally leaning towards option 2: it is nearly backwards compatible, except for obscure cases where people have mbcs-encodable entries in sys.path already, and it is independent of manipulations of the system encoding. I also think that processing of PYTHONPATH should take Unicode into account, i.e. we should use _wgetenv to access PYTHONPATH in 2.6. That would make the feature truly useful, as then people could actually set sys.path to non-mbcs directlories from the outside. Notice that W9x support can be dropped in 2.6, so a W9x-compatible solution won't be required. In any case, I'd like to encourage you to continue working on this issue. I, too, like to see it in 2.6, but I did so ever since 2.1 or so (before PEP 277 was implemented), and it was wishful thinking. Somebody has to take action, and it is likely that it won't one of the past regular contributors (or else they had contributed it long ago - although I think Thomas Heller had something working at one point). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2006-09-12 11:29 Message: Logged In: YES user_id=29957 There's a variety of modules in the standard library that reference __file__ - if it's potentially going to be a unicode string, these are going to need to be checked, as are their callers :-/ (Now that I've looked closer at some of the issues, I'm extremely glad this didn't go into 2.5 final at this late stage) ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2006-09-12 09:38 Message: Logged In: YES user_id=1262199 I submitted this mostly as a demonstration. I don't think the approach is necessarily suitable for a final implementation because of the use of utf-8 as an intermediate representation and the price of the conversions that keep happening. But perhaps this is the way to go, if we consider utf-8 to be a stage-1 default file system encoding for win32. I also agree that 4 is probably the most sensible approach. What about discrepancies between e.g. linux and windows then, when including from a non-trivial path? On linux we would get utf-8, on windows unicode? 1) would actually make a lot of sense, only in my experience this tends to lead to a kind of unicode-hell since a program touched by one unicode object tends to have it percolating down into every corner. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-09-09 12:31 Message: Logged In: YES user_id=21627 First: Do you want to continue to work on this, or do you consider this just "demonstration code" (i.e. not contributed for inclusion in Python), hoping that somebody else implements this feature? I think the behavior of __file__ must be more consistent across platforms, and the selected behaviour must be documented somewhere. Several definitions of "consistent behavior" come to mind: 1. __file__ is always a Unicode string 2. __file__ is a byte string if its ASCII, else Unicode 3. __file__ is a byte string if its in the system encoding, else Unicode 4. __file__ is a byte string if its in the file system encoding, else Unicode. The documentation needs to be updated in several places, e.g. also for inspect.getfile. I would expect that pydoc would also need to be updated. Selecting from the options above: I believe 4 is most compatible with previous versions; 1 and 2 are most convenient to work with in applications like pydoc which have to generate HTML (1 is easier to work with, 2 is more compatible with previous versions). ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2006-09-09 11:38 Message: Logged In: YES user_id=1262199 >From the top of my head, it is now unicode. I consider trying to convert it back to the default encoding but decided not to to keep the patch brief. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-09-08 21:03 Message: Logged In: YES user_id=21627 What is the value of the __file__ attribute of a module when this patch is used? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1552880&group_id=5470 From noreply at sourceforge.net Wed Apr 18 05:51:19 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 17 Apr 2007 20:51:19 -0700 Subject: [Patches] [ python-Patches-1685642 ] Add IllegalStateError Message-ID: Patches item #1685642, was opened at 2007-03-21 20:25 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685642&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None >Group: Python 3000 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Bj?rn Lindqvist (sonderblade) Assigned to: Guido van Rossum (gvanrossum) Summary: Add IllegalStateError Initial Comment: Java has an exception called IllegalStateException. It is very useful for signaling "what you did would have been OK, but right now I can't do it." Python does not have a counterpart, but I think it should. For example, if you have a Python RDBMS-binding and you call db.select() before calling db.connect() it could raise a IllegalStateError (or maybe a subclass of it) that says 'IllegalStateError: "Must be connected to perform queries"'. Or for another example, take the Thread.start() function. Currently, if you call myThread.start() twice it will raise an AssertionError. That's no good because the result is undefined if you run python with -O. See Bug #904498 and Patch #1676820 (Martin v. L??wis comment). IllegalStateError should fit perfectly here: >>> t = threading.Thread() >>> t.start() >>> t.start() Traceback (most recent call last): ... IllegalStateError: thread already started ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-17 23:51 Message: Logged In: YES user_id=6380 Originator: NO Following Raymond's advice, I'm rejecting this. We don't need a lot of different standard exceptions for use by application code; it's better for libraries and modules to define their own exceptions. RuntimeError is already a compromise -- it's mostly meant to save quick-and-dirty code from having to define exceptions, and to avoid the mistake of raising Exception(). (Some use of RuntimeError in the standard library notwithstanding.) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2007-04-11 13:36 Message: Logged In: YES user_id=80475 Originator: NO Guido, would you pronounce on this? I think the existing framework is fine and adding a new standard exception would be unnecessary clutter. ---------------------------------------------------------------------- Comment By: Bj?rn Lindqvist (sonderblade) Date: 2007-03-26 12:29 Message: Logged In: YES user_id=51702 Originator: YES Georg, do you think that I should change the patch so that IllegalStateError sublcasses RuntimeError instead? Changing exceptions in the Standard Library to IllegalStateError would be part 2 of the patch. I don't think that should cause any trouble - only exceptions like AssertionError or base Exceptions would be changed. That's why I hope it can go into python 2.x. But bringing it up on the mailing-list is still required I guess. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 14:41 Message: Logged In: YES user_id=849994 Originator: NO In principle this is a good idea -- currently there's no guideline what to raise in such a situation. Some libraries raise RuntimeError (probably a new exception should be made subclass thereof), some do assertions, some raise custom exceptions. However, for this to be a complete patch, you'd have to find all those places where IllegalStateError would be appropriate. Also, switching to it would be a Py3k thing, provided that this breakage is suffered by python-dev. In any case, I think you should bring it up on the python-3000 mailing list -- otherwise this will never be decided upon. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1685642&group_id=5470 From noreply at sourceforge.net Wed Apr 18 18:02:43 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 09:02:43 -0700 Subject: [Patches] [ python-Patches-1686487 ] Allow any mapping after ** in calls Message-ID: Patches item #1686487, was opened at 2007-03-22 23:27 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: Allow any mapping after ** in calls Initial Comment: Python allows arbitrary sequences after * in calls, but an expression following ** must be a (subclass of) dict. The attached patch makes Python accept arbitrary mappings after **. ---------------------------------------------------------------------- >Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 12:02 Message: Logged In: YES user_id=835142 Originator: YES I have asked on python-dev almost a month ago, but did not get any response: http://mail.python.org/pipermail/python-dev/2007-March/072321.html I have recently posted another message that got no response as well: http://mail.python.org/pipermail/python-dev/2007-April/072653.html Is it possible that my messages are not being delivered to the list? ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-03-23 15:44 Message: Logged In: YES user_id=835142 Originator: YES I've fixed the flaws in star-star-mapping-1.patch . I will ask on python-dev about the feature. If the proposal is accepted, I will probably refactor the code to avoid using the second goto label. File Added: star-star-mapping-1.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 14:45 Message: Logged In: YES user_id=849994 Originator: NO Two flaws: - what if kwdict is NULL? Shouldn't happen, but the old version raises an error in this case. - if PyDict_Update fails, tmpdict is leaked. Please ask on python-dev whether to include this feature. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 From noreply at sourceforge.net Wed Apr 18 19:45:44 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 10:45:44 -0700 Subject: [Patches] [ python-Patches-1671298 ] Refactor test_class to use unittest lib Message-ID: Patches item #1671298, was opened at 2007-02-28 15:38 Message generated for change (Comment added) made by jseutter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1671298&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mike Verdone (jyzude) Assigned to: Collin Winter (collinwinter) Summary: Refactor test_class to use unittest lib Initial Comment: Refactored Lib/test/test_class.py to use unittest library instead of icky output comparison tests. Also have to delete output/test_class after adding this patch. ---------------------------------------------------------------------- Comment By: Jerry Seutter (jseutter) Date: 2007-04-18 11:45 Message: Logged In: YES user_id=1727609 Originator: NO This patch looks good to me. ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-19 16:23 Message: Logged In: YES user_id=584997 Originator: YES Hi Collin, Sorry for the delay. Your tweaks look very good. I don't see any problems. ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-11 11:42 Message: Logged In: YES user_id=1344176 Originator: NO Mike, I've tweaked your refactoring some: strengthened the assertions in testMixIntsAndLongs and testDel; code cleanup in testBadTypeReturned; removed Jython-related code; reduced line lengths to < 80; changed a few print statements to self.fail() calls. Look over the version I've uploaded and tell me what you think. File Added: test_class.py.diff ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-07 14:07 Message: Logged In: YES user_id=584997 Originator: YES Removed unnecessary global statement in trackCall. Anything else? :-) File Added: test_class.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-06 18:03 Message: Logged In: YES user_id=849994 Originator: NO Note that you don't need the global statement for callLst as you aren't rebinding it in the function. ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-05 23:38 Message: Logged In: YES user_id=584997 Originator: YES Hi collin, * I improved testMixIntsAndLongs. It now asserts things * assert is banished, replaced by the correct calls * the reason why callLst is global is because I have to track calls to __getitem__ in some cases. Because of this, if I put callLst on the object I end up with horrible recursive loops, or at the very least the last call on the stack will always be __getitem__ when I get the list to inspect. * assertLastCall only checks the last thing on the list because generally the thing called before that is always __getitem__ or associated magic. I don't want my tests to be bound to the internals of __getitem__. All I care about is that ultimately the right function was called. That said, I modified assertLastCallWas to erase the callLst to prevent any possible bleed-over from the previous test. Let me know if you have further suggestions. File Added: test_class.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-05 17:11 Message: Logged In: YES user_id=1344176 Originator: NO Thanks for your effort! This generally looks good. A few minor things: - testMixIntsAndLongs doesn't have any assertions in it. - "assert" statements should probably be changed to use the failUnlessEqual/assertEqual methods. - I'm wary of your assertLastCallWas system; I'd feel more comfortable if you were making assertions about the entire call chain, not just its last item. Also, something like callLst feels strange as a global. Feel free to contact me to discuss this off-list. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1671298&group_id=5470 From noreply at sourceforge.net Wed Apr 18 20:34:05 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 11:34:05 -0700 Subject: [Patches] [ python-Patches-1686487 ] Allow any mapping after ** in calls Message-ID: Patches item #1686487, was opened at 2007-03-22 23:27 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: Allow any mapping after ** in calls Initial Comment: Python allows arbitrary sequences after * in calls, but an expression following ** must be a (subclass of) dict. The attached patch makes Python accept arbitrary mappings after **. ---------------------------------------------------------------------- >Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 14:34 Message: Logged In: YES user_id=835142 Originator: YES Georg, Thanks for reposting my proposal. Since BDFL is +1 on the idea, I feel motivated to polish the patch. In your early comment you noted that kwdict == NULL check may be superficial. I agree. It does not look like it is possible that ext_do_call is called with CALL_FLAG_KW set and a null pointer at the top of the stack. It looks like this may only happen if the stack is corrupted by a misbehaving c extension. In this case, however, throwing a type error is too gentle. PyErr_BadInternalCall() seems to be more appropriate if anything at all. I propose to eliminate the null check for kwdict alltogether. This would be consistent with the way stararg is handled later in this function. I am going to run the testsuit without the null check and will post a revised patch shortly. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 12:02 Message: Logged In: YES user_id=835142 Originator: YES I have asked on python-dev almost a month ago, but did not get any response: http://mail.python.org/pipermail/python-dev/2007-March/072321.html I have recently posted another message that got no response as well: http://mail.python.org/pipermail/python-dev/2007-April/072653.html Is it possible that my messages are not being delivered to the list? ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-03-23 15:44 Message: Logged In: YES user_id=835142 Originator: YES I've fixed the flaws in star-star-mapping-1.patch . I will ask on python-dev about the feature. If the proposal is accepted, I will probably refactor the code to avoid using the second goto label. File Added: star-star-mapping-1.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 14:45 Message: Logged In: YES user_id=849994 Originator: NO Two flaws: - what if kwdict is NULL? Shouldn't happen, but the old version raises an error in this case. - if PyDict_Update fails, tmpdict is leaked. Please ask on python-dev whether to include this feature. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 From noreply at sourceforge.net Wed Apr 18 21:03:11 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 12:03:11 -0700 Subject: [Patches] [ python-Patches-1703178 ] link_objects in setup.cfg crashes build Message-ID: Patches item #1703178, was opened at 2007-04-18 21:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703178&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils and setup.py Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Markus Schatten (mschatte) Assigned to: Nobody/Anonymous (nobody) Summary: link_objects in setup.cfg crashes build Initial Comment: When I put the following into setup.cfg: --- [build_ext] include_dirs = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu link_objects = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o --- in order to link a c extension to another object which should reside on the users computer, I get the following error (tested on two kubuntu machines, python 2.4): # python setup.py build Distribution.parse_config_files(): reading setup.cfg options (after parsing config files): option dict for 'build_ext' command: {'include_dirs': ('setup.cfg', '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu'), 'link_objects': ('setup.cfg', '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o')} options (after parsing command line): option dict for 'build' command: {} option dict for 'build_ext' command: {'include_dirs': ('setup.cfg', '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu'), 'link_objects': ('setup.cfg', '/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o')} running build Distribution.get_command_obj(): creating 'build' command object running build_py Distribution.get_command_obj(): creating 'build_py' command object creating build creating build/lib.linux-i686-2.4 copying dbms.py -> build/lib.linux-i686-2.4 copying __init__.py -> build/lib.linux-i686-2.4 copying f.py -> build/lib.linux-i686-2.4 copying interface.py -> build/lib.linux-i686-2.4 copying py2f.py -> build/lib.linux-i686-2.4 copying reasonable.py -> build/lib.linux-i686-2.4 creating build/lib.linux-i686-2.4/xsb_swig copying xsb_swig/__init__.py -> build/lib.linux-i686-2.4/xsb_swig copying xsb_swig/xsb.py -> build/lib.linux-i686-2.4/xsb_swig running build_ext Distribution.get_command_obj(): creating 'build_ext' command object setting options for 'build_ext' command: link_objects = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu/saved.o/xsb.o (from setup.cfg) include_dirs = /home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu:/home/mschatte/myPackages/XSB/emu (from setup.cfg) building 'xsb_swig._xsb' extension creating build/temp.linux-i686-2.4 creating build/temp.linux-i686-2.4/xsb_swig gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -fPIC -I/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu -I/home/mschatte/myPackages/XSB/emu -I/usr/include/python2.4 -c xsb_swig/swig_xsb_wrap.c -o build/temp.linux-i686-2.4/xsb_swig/swig_xsb_wrap.o In file included from xsb_swig/swig_xsb.h:25, from xsb_swig/swig_xsb_wrap.c:2346: /home/mschatte/myPackages/XSB/emu/cinterf.h:259: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:262: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:285: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:286: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:294: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:296: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:300: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:304: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:305: warning: function declaration isn?t a prototype In file included from xsb_swig/swig_xsb_wrap.c:2346: xsb_swig/swig_xsb.h:101: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:135: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:137: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:141: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:145: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:146: warning: function declaration isn?t a prototype xsb_swig/swig_xsb_wrap.c:2498: warning: function declaration isn?t a prototype xsb_swig/swig_xsb_wrap.c:5124: warning: function declaration isn?t a prototype gcc -pthread -fno-strict-aliasing -DNDEBUG -g -O2 -Wall -Wstrict-prototypes -fPIC -I/home/mschatte/myPackages/XSB/config/i686-pc-linux-gnu -I/home/mschatte/myPackages/XSB/emu -I/usr/include/python2.4 -c xsb_swig/swig_xsb.c -o build/temp.linux-i686-2.4/xsb_swig/swig_xsb.o In file included from xsb_swig/swig_xsb.h:25, from xsb_swig/swig_xsb.c:28: /home/mschatte/myPackages/XSB/emu/cinterf.h:259: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:262: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:285: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:286: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:294: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:296: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:300: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:304: warning: function declaration isn?t a prototype /home/mschatte/myPackages/XSB/emu/cinterf.h:305: warning: function declaration isn?t a prototype In file included from xsb_swig/swig_xsb.c:28: xsb_swig/swig_xsb.h:101: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:135: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:137: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:141: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:145: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.h:146: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.c: In function ?intpointer?: xsb_swig/swig_xsb.c:84: warning: function returns address of local variable xsb_swig/swig_xsb.c: At top level: xsb_swig/swig_xsb.c:140: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.c:174: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.c:176: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.c:180: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.c:184: warning: function declaration isn?t a prototype xsb_swig/swig_xsb.c:185: warning: function declaration isn?t a prototype Traceback (most recent call last): File "setup.py", line 11, in ? [ "xsb_swig/swig_xsb_wrap.c", "xsb_swig/swig_xsb.c" ] File "distutils/core.py", line 149, in setup File "distutils/dist.py", line 946, in run_commands File "distutils/dist.py", line 966, in run_command File "distutils/command/build.py", line 112, in run File "/usr/lib/python2.4/cmd.py", line 333, in run_command del help[cmd] File "distutils/dist.py", line 966, in run_command File "distutils/command/build_ext.py", line 279, in run File "distutils/command/build_ext.py", line 405, in build_extensions File "distutils/command/build_ext.py", line 502, in build_extension File "distutils/ccompiler.py", line 847, in link_shared_object File "distutils/unixccompiler.py", line 211, in link TypeError: can only concatenate list (not "str") to list --- maybe it isn't a bug, but link_objects is not well documented, so I wasn't able to find the appropriate syntax for setup.cfg Best regards, -- Markus Schatten, dipl. inf. e-mail: markus.schatten at foi.hr Faculty of Organization and Informatics Vara?din, Croatia ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703178&group_id=5470 From noreply at sourceforge.net Wed Apr 18 21:38:58 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 12:38:58 -0700 Subject: [Patches] [ python-Patches-1686487 ] Allow any mapping after ** in calls Message-ID: Patches item #1686487, was opened at 2007-03-22 23:27 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: Allow any mapping after ** in calls Initial Comment: Python allows arbitrary sequences after * in calls, but an expression following ** must be a (subclass of) dict. The attached patch makes Python accept arbitrary mappings after **. ---------------------------------------------------------------------- >Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 15:38 Message: Logged In: YES user_id=835142 Originator: YES The proposed patch handles ** arguments similar to the way * arguments are treated. If the expression following ** evaluates to an object that is not a dictionary, the new code attempts to update a new dictionary with that object. This procedure is closely similar to what PySequence_Tuple does with * arguments. (Python API does not provide PyMapping_Dict.) If new dictionary is succesfully created and updated, it replaces the original ** argument. If that attempt fails because kwdict is not a mapping (does not have a 'keys' attribute, a type error is raised explaining that a mapping is expected. All other errors, e.g. MemoryError are percolated up. (I am am removing the previous versions of the patch.) File Added: star-star-mapping.patch ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 14:34 Message: Logged In: YES user_id=835142 Originator: YES Georg, Thanks for reposting my proposal. Since BDFL is +1 on the idea, I feel motivated to polish the patch. In your early comment you noted that kwdict == NULL check may be superficial. I agree. It does not look like it is possible that ext_do_call is called with CALL_FLAG_KW set and a null pointer at the top of the stack. It looks like this may only happen if the stack is corrupted by a misbehaving c extension. In this case, however, throwing a type error is too gentle. PyErr_BadInternalCall() seems to be more appropriate if anything at all. I propose to eliminate the null check for kwdict alltogether. This would be consistent with the way stararg is handled later in this function. I am going to run the testsuit without the null check and will post a revised patch shortly. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 12:02 Message: Logged In: YES user_id=835142 Originator: YES I have asked on python-dev almost a month ago, but did not get any response: http://mail.python.org/pipermail/python-dev/2007-March/072321.html I have recently posted another message that got no response as well: http://mail.python.org/pipermail/python-dev/2007-April/072653.html Is it possible that my messages are not being delivered to the list? ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-03-23 15:44 Message: Logged In: YES user_id=835142 Originator: YES I've fixed the flaws in star-star-mapping-1.patch . I will ask on python-dev about the feature. If the proposal is accepted, I will probably refactor the code to avoid using the second goto label. File Added: star-star-mapping-1.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 14:45 Message: Logged In: YES user_id=849994 Originator: NO Two flaws: - what if kwdict is NULL? Shouldn't happen, but the old version raises an error in this case. - if PyDict_Update fails, tmpdict is leaked. Please ask on python-dev whether to include this feature. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 From noreply at sourceforge.net Wed Apr 18 21:53:07 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 12:53:07 -0700 Subject: [Patches] [ python-Patches-1686487 ] Allow any mapping after ** in calls Message-ID: Patches item #1686487, was opened at 2007-03-22 23:27 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: Allow any mapping after ** in calls Initial Comment: Python allows arbitrary sequences after * in calls, but an expression following ** must be a (subclass of) dict. The attached patch makes Python accept arbitrary mappings after **. ---------------------------------------------------------------------- >Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 15:53 Message: Logged In: YES user_id=835142 Originator: YES File Added: star-star-mapping.patch ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 15:38 Message: Logged In: YES user_id=835142 Originator: YES The proposed patch handles ** arguments similar to the way * arguments are treated. If the expression following ** evaluates to an object that is not a dictionary, the new code attempts to update a new dictionary with that object. This procedure is closely similar to what PySequence_Tuple does with * arguments. (Python API does not provide PyMapping_Dict.) If new dictionary is succesfully created and updated, it replaces the original ** argument. If that attempt fails because kwdict is not a mapping (does not have a 'keys' attribute, a type error is raised explaining that a mapping is expected. All other errors, e.g. MemoryError are percolated up. (I am am removing the previous versions of the patch.) File Added: star-star-mapping.patch ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 14:34 Message: Logged In: YES user_id=835142 Originator: YES Georg, Thanks for reposting my proposal. Since BDFL is +1 on the idea, I feel motivated to polish the patch. In your early comment you noted that kwdict == NULL check may be superficial. I agree. It does not look like it is possible that ext_do_call is called with CALL_FLAG_KW set and a null pointer at the top of the stack. It looks like this may only happen if the stack is corrupted by a misbehaving c extension. In this case, however, throwing a type error is too gentle. PyErr_BadInternalCall() seems to be more appropriate if anything at all. I propose to eliminate the null check for kwdict alltogether. This would be consistent with the way stararg is handled later in this function. I am going to run the testsuit without the null check and will post a revised patch shortly. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 12:02 Message: Logged In: YES user_id=835142 Originator: YES I have asked on python-dev almost a month ago, but did not get any response: http://mail.python.org/pipermail/python-dev/2007-March/072321.html I have recently posted another message that got no response as well: http://mail.python.org/pipermail/python-dev/2007-April/072653.html Is it possible that my messages are not being delivered to the list? ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-03-23 15:44 Message: Logged In: YES user_id=835142 Originator: YES I've fixed the flaws in star-star-mapping-1.patch . I will ask on python-dev about the feature. If the proposal is accepted, I will probably refactor the code to avoid using the second goto label. File Added: star-star-mapping-1.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 14:45 Message: Logged In: YES user_id=849994 Originator: NO Two flaws: - what if kwdict is NULL? Shouldn't happen, but the old version raises an error in this case. - if PyDict_Update fails, tmpdict is leaked. Please ask on python-dev whether to include this feature. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 From noreply at sourceforge.net Wed Apr 18 23:37:03 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 14:37:03 -0700 Subject: [Patches] [ python-Patches-1703268 ] silenced a compiler warning Message-ID: Patches item #1703268, was opened at 2007-04-18 17:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703268&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: silenced a compiler warning Initial Comment: Gcc complains as follows: Objects/frameobject.c:140: warning: passing arg 2 of `PyString_AsStringAndSize' from incompatible pointer type because code is declared as unsigned char* and PyString_AsStringAndSize expects an address of a plain char*. Attached patch adds an explicit cast to silence the warning. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703268&group_id=5470 From noreply at sourceforge.net Wed Apr 18 23:43:45 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 14:43:45 -0700 Subject: [Patches] [ python-Patches-1703270 ] missing declaration in readline.c Message-ID: Patches item #1703270, was opened at 2007-04-18 17:43 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703270&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: missing declaration in readline.c Initial Comment: If HAVE_RL_COMPLETION_MATCHES is not defined, readline.c code does not define completion_matches macro in an apparent hope that completion_matches will be linked to a n internal rl function. Attached patch adds the missing extern declaration to supress compiler warnings. Currently gcc complains as follows: Modules/readline.c:681: warning: implicit declaration of function `completion_matches' Modules/readline.c:681: warning: return makes pointer from integer without a cast ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703270&group_id=5470 From noreply at sourceforge.net Wed Apr 18 23:54:17 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 14:54:17 -0700 Subject: [Patches] [ python-Patches-1675363 ] PEP 3114 -- next() -> __next__() Message-ID: Patches item #1675363, was opened at 2007-03-06 19:21 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1675363&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) >Assigned to: Guido van Rossum (gvanrossum) Summary: PEP 3114 -- next() -> __next__() Initial Comment: This patch implements PEP 3114. It renames all next() methods to __next__, introduces the new builtin next() with one or two arguments and updates the documentation accordingly. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-18 17:54 Message: Logged In: YES user_id=6380 Originator: NO Can you bring this up to date and commit it? ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-07 06:27 Message: Logged In: YES user_id=849994 Originator: YES Version 2 adds tests for the new builtin next(). File Added: __next__.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1675363&group_id=5470 From noreply at sourceforge.net Thu Apr 19 00:09:31 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 15:09:31 -0700 Subject: [Patches] [ python-Patches-1703286 ] bug fix: ctypes truncates 64-bit pointers Message-ID: Patches item #1703286, was opened at 2007-04-18 18:09 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: bug fix: ctypes truncates 64-bit pointers Initial Comment: At several places _ctypes module converts a python integer to a void* pointer by first extracting an argument into an int. Attached patch changes int to Py_ssize_t that is guaranteed to be wide enough to hold a pointer. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 From noreply at sourceforge.net Thu Apr 19 00:29:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 15:29:54 -0700 Subject: [Patches] [ python-Patches-1703300 ] fixes non ansi c declarations in libffi Message-ID: Patches item #1703300, was opened at 2007-04-18 18:29 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703300&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: fixes non ansi c declarations in libffi Initial Comment: attached patch fixes compiler warnings Modules/_ctypes/libffi/src/x86/ffi64.c:45: warning: function declaration isn't a prototype Modules/_ctypes/libffi/src/x86/ffi64.c:342: warning: function declaration isn't a prototype from libffi on x86_64 platform. Similar changes should be made for all platforms and submitted upstream (to gcc?). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703300&group_id=5470 From noreply at sourceforge.net Thu Apr 19 05:49:51 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 20:49:51 -0700 Subject: [Patches] [ python-Patches-1703379 ] Refactor test_frozen.py to use unittest. Message-ID: Patches item #1703379, was opened at 2007-04-18 21:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jerry Seutter (jseutter) Assigned to: Nobody/Anonymous (nobody) Summary: Refactor test_frozen.py to use unittest. Initial Comment: This patch refactors test_frozen.py to use unittest. When this patch is applied, Lib/test/output/test_frozen can be removed. I am not sure if the last test case is named appropriately. I named it testImportNonexistentFunction because I believe the code is trying to import a function __phello__.foo, which does not exist. I may be mistaken. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 From noreply at sourceforge.net Thu Apr 19 07:26:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 22:26:54 -0700 Subject: [Patches] [ python-Patches-1703268 ] silenced a compiler warning Message-ID: Patches item #1703268, was opened at 2007-04-18 14:37 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703268&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: silenced a compiler warning Initial Comment: Gcc complains as follows: Objects/frameobject.c:140: warning: passing arg 2 of `PyString_AsStringAndSize' from incompatible pointer type because code is declared as unsigned char* and PyString_AsStringAndSize expects an address of a plain char*. Attached patch adds an explicit cast to silence the warning. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 22:26 Message: Logged In: YES user_id=33168 Originator: NO Brett just checked in 54873 to fix this problem. Thanks for the patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703268&group_id=5470 From noreply at sourceforge.net Thu Apr 19 07:53:04 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 22:53:04 -0700 Subject: [Patches] [ python-Patches-1703270 ] missing declaration in readline.c Message-ID: Patches item #1703270, was opened at 2007-04-18 14:43 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703270&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) >Assigned to: Neal Norwitz (nnorwitz) Summary: missing declaration in readline.c Initial Comment: If HAVE_RL_COMPLETION_MATCHES is not defined, readline.c code does not define completion_matches macro in an apparent hope that completion_matches will be linked to a n internal rl function. Attached patch adds the missing extern declaration to supress compiler warnings. Currently gcc complains as follows: Modules/readline.c:681: warning: implicit declaration of function `completion_matches' Modules/readline.c:681: warning: return makes pointer from integer without a cast ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 22:53 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! Committed revision 54874. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703270&group_id=5470 From noreply at sourceforge.net Thu Apr 19 07:54:46 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 22:54:46 -0700 Subject: [Patches] [ python-Patches-1703300 ] fixes non ansi c declarations in libffi Message-ID: Patches item #1703300, was opened at 2007-04-18 15:29 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703300&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) >Assigned to: Thomas Heller (theller) Summary: fixes non ansi c declarations in libffi Initial Comment: attached patch fixes compiler warnings Modules/_ctypes/libffi/src/x86/ffi64.c:45: warning: function declaration isn't a prototype Modules/_ctypes/libffi/src/x86/ffi64.c:342: warning: function declaration isn't a prototype from libffi on x86_64 platform. Similar changes should be made for all platforms and submitted upstream (to gcc?). ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 22:54 Message: Logged In: YES user_id=33168 Originator: NO Thomas, could you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703300&group_id=5470 From noreply at sourceforge.net Thu Apr 19 07:56:35 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 22:56:35 -0700 Subject: [Patches] [ python-Patches-1703286 ] bug fix: ctypes truncates 64-bit pointers Message-ID: Patches item #1703286, was opened at 2007-04-18 15:09 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: bug fix: ctypes truncates 64-bit pointers Initial Comment: At several places _ctypes module converts a python integer to a void* pointer by first extracting an argument into an int. Attached patch changes int to Py_ssize_t that is guaranteed to be wide enough to hold a pointer. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 22:56 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch. The changes look correct, however, can you create a test for these cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:08:28 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:08:28 -0700 Subject: [Patches] [ python-Patches-1703379 ] Refactor test_frozen.py to use unittest. Message-ID: Patches item #1703379, was opened at 2007-04-18 20:49 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jerry Seutter (jseutter) >Assigned to: Ronald Oussoren (ronaldoussoren) Summary: Refactor test_frozen.py to use unittest. Initial Comment: This patch refactors test_frozen.py to use unittest. When this patch is applied, Lib/test/output/test_frozen can be removed. I am not sure if the last test case is named appropriately. I named it testImportNonexistentFunction because I believe the code is trying to import a function __phello__.foo, which does not exist. I may be mistaken. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:08 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! It looks pretty good, but there is one problem I see. There should be a test_main() function for this to work from regrtest. For example, take a look in Lib/test/test_bool.py. Also, I don't think you need to seek(0) before getting the value from sys.stdout (ie, cStringIO instance). I don't think you need to support sys.platform == 'mac'. That was for Mac OS 9 which is no longer supported (see PEP 11). Mac OS 10 has sys.platform == 'darwin'. If you're interested, it would be good to produce a patch to remove all (10+) occurrences of comparing sys.platform with mac. I'm assigning this to Ronald just so he can verify the Mac OS 9 issue. Ronald, please comment and then unassign or assign to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:17:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:17:23 -0700 Subject: [Patches] [ python-Patches-1671298 ] Refactor test_class to use unittest lib Message-ID: Patches item #1671298, was opened at 2007-02-28 14:38 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1671298&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mike Verdone (jyzude) Assigned to: Collin Winter (collinwinter) Summary: Refactor test_class to use unittest lib Initial Comment: Refactored Lib/test/test_class.py to use unittest library instead of icky output comparison tests. Also have to delete output/test_class after adding this patch. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:17 Message: Logged In: YES user_id=33168 Originator: NO There should be a test_main method otherwise this won't work when called from regrtest. Have you tried to run this with regrtest -R :: ? I think it will work, but just wanted to be sure. One thing that might be nice to add to this test is to verify the change in the length of the callLst since assertLastCallWas() was called. Typically there should be only one method call from what I saw in the test. However, if we screw something up and there are two method calls, this test could catch that. It would be an enhancement (ie new feature) over the existing test. ---------------------------------------------------------------------- Comment By: Jerry Seutter (jseutter) Date: 2007-04-18 10:45 Message: Logged In: YES user_id=1727609 Originator: NO This patch looks good to me. ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-19 15:23 Message: Logged In: YES user_id=584997 Originator: YES Hi Collin, Sorry for the delay. Your tweaks look very good. I don't see any problems. ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-11 10:42 Message: Logged In: YES user_id=1344176 Originator: NO Mike, I've tweaked your refactoring some: strengthened the assertions in testMixIntsAndLongs and testDel; code cleanup in testBadTypeReturned; removed Jython-related code; reduced line lengths to < 80; changed a few print statements to self.fail() calls. Look over the version I've uploaded and tell me what you think. File Added: test_class.py.diff ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-07 13:07 Message: Logged In: YES user_id=584997 Originator: YES Removed unnecessary global statement in trackCall. Anything else? :-) File Added: test_class.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-06 17:03 Message: Logged In: YES user_id=849994 Originator: NO Note that you don't need the global statement for callLst as you aren't rebinding it in the function. ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-05 22:38 Message: Logged In: YES user_id=584997 Originator: YES Hi collin, * I improved testMixIntsAndLongs. It now asserts things * assert is banished, replaced by the correct calls * the reason why callLst is global is because I have to track calls to __getitem__ in some cases. Because of this, if I put callLst on the object I end up with horrible recursive loops, or at the very least the last call on the stack will always be __getitem__ when I get the list to inspect. * assertLastCall only checks the last thing on the list because generally the thing called before that is always __getitem__ or associated magic. I don't want my tests to be bound to the internals of __getitem__. All I care about is that ultimately the right function was called. That said, I modified assertLastCallWas to erase the callLst to prevent any possible bleed-over from the previous test. Let me know if you have further suggestions. File Added: test_class.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-05 16:11 Message: Logged In: YES user_id=1344176 Originator: NO Thanks for your effort! This generally looks good. A few minor things: - testMixIntsAndLongs doesn't have any assertions in it. - "assert" statements should probably be changed to use the failUnlessEqual/assertEqual methods. - I'm wary of your assertLastCallWas system; I'd feel more comfortable if you were making assertions about the entire call chain, not just its last item. Also, something like callLst feels strange as a global. Feel free to contact me to discuss this off-list. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1671298&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:18:26 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:18:26 -0700 Subject: [Patches] [ python-Patches-1703379 ] Refactor test_frozen.py to use unittest. Message-ID: Patches item #1703379, was opened at 2007-04-19 05:49 Message generated for change (Comment added) made by ronaldoussoren You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jerry Seutter (jseutter) >Assigned to: Neal Norwitz (nnorwitz) Summary: Refactor test_frozen.py to use unittest. Initial Comment: This patch refactors test_frozen.py to use unittest. When this patch is applied, Lib/test/output/test_frozen can be removed. I am not sure if the last test case is named appropriately. I named it testImportNonexistentFunction because I believe the code is trying to import a function __phello__.foo, which does not exist. I may be mistaken. ---------------------------------------------------------------------- >Comment By: Ronald Oussoren (ronaldoussoren) Date: 2007-04-19 08:18 Message: Logged In: YES user_id=580910 Originator: NO As Neal wrote sys.platform == 'mac' is of OS9, which isn't supported anymore by Python (or Apple for that matter). The special-case for platform == 'mac' should be removed altoghether and not replaced by a check for OSX because OSX is basically just another unix platform and should therefore support freezing. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 08:08 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! It looks pretty good, but there is one problem I see. There should be a test_main() function for this to work from regrtest. For example, take a look in Lib/test/test_bool.py. Also, I don't think you need to seek(0) before getting the value from sys.stdout (ie, cStringIO instance). I don't think you need to support sys.platform == 'mac'. That was for Mac OS 9 which is no longer supported (see PEP 11). Mac OS 10 has sys.platform == 'darwin'. If you're interested, it would be good to produce a patch to remove all (10+) occurrences of comparing sys.platform with mac. I'm assigning this to Ronald just so he can verify the Mac OS 9 issue. Ronald, please comment and then unassign or assign to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:22:04 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:22:04 -0700 Subject: [Patches] [ python-Patches-1698915 ] ZipFile.printdir fix (2.5) Message-ID: Patches item #1698915, was opened at 2007-04-11 20:22 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698915&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Duplicate Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.5) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:22 Message: Logged In: YES user_id=33168 Originator: NO Closing this as a dupe of 1698917. They are the same patch for either 2.5 or 2.6. Either patch should apply cleanly to both the trunk and 2.5 branch (which this fix should applied to). ---------------------------------------------------------------------- Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-11 20:27 Message: Logged In: YES user_id=1115903 Originator: YES I forgot to include that this patch passes regrtest.py -r ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698915&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:22:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:22:54 -0700 Subject: [Patches] [ python-Patches-1698917 ] ZipFile.printdir fix (2.6) Message-ID: Patches item #1698917, was opened at 2007-04-11 20:27 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.6) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). Passes full regrtest.py -r ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:22 Message: Logged In: YES user_id=33168 Originator: NO Alan, thanks for the patch. Would you create a test case for this? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:29:40 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:29:40 -0700 Subject: [Patches] [ python-Patches-1691032 ] Migrate test_minidom.py to unittest Message-ID: Patches item #1691032, was opened at 2007-03-29 21:15 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: Migrate test_minidom.py to unittest Initial Comment: Update Lib/test/test_minidom.py to use the unittest module. Also: I noticed the test_minidom.py contains some code (now long dead) to check for reference cycles. The code is active only when Node.allnodes exists; this hasn't existed for over four years now. After re-implementing that test using gc.DEBUG_SAVEALL, I found that many tests failed because of missing unlink() calls. I added them. After that, one test was still failing. This is because expatbuilder.parseString() and friends do not clean up some cyclic references on error. The patch includes a fix for that (in Lib/xml/dom/expatbuilder.py). An alternative would be to ditch the cycle-checking test entirely. I can submit a patch that does that instead, if desired. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:29 Message: Logged In: YES user_id=33168 Originator: NO Jason, can you clarify your last comment. Wrt the "change [not] worth making", do you mean this patch or a different change? I just want to know what in this patch is still active and needs review before spending time. Thanks. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-04 19:01 Message: Logged In: YES user_id=18139 Originator: YES Collin: Thanks for taking the time to look at this. I don't think this change is worth making. I've started working on some new features in minidom and it seems like every time I exercise a new piece of code, the gc finds a small number of objects in cycles. For example, parseFragmentString() seems to leave a few objects (even with expatbuilder.second.patch). We're just going to have to live with it-- I don't think anyone has time or desire to hunt down them all down. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-01 14:18 Message: Logged In: YES user_id=18139 Originator: YES File Added: expatbuilder.second.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-01 14:18 Message: Logged In: YES user_id=18139 Originator: YES OK, I've attached two new patches. test_minidom.second.patch - Changes "confirm" to "assert_" or "assertEquals", breaks multi-line assertions into many one-line assertions, adds convenience methods "assertSameNode" and "assertNotSameNode", etc. Also replaces the Node.allnodes test with gc-based test and adds a few calls to unlink(). expatbuilder.second.patch - Fixes the test failure discovered by the new gc testing. File Added: test_minidom.second.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-29 23:49 Message: Logged In: YES user_id=1344176 Originator: NO test_minidom already uses unittest (as of patch #1683397, SVN r54603). Could you update your patch to the latest trunk revision? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:37:38 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:37:38 -0700 Subject: [Patches] [ python-Patches-1691070 ] Speed up PyArg_ParseTupleAndKeywords() and improve error msg Message-ID: Patches item #1691070, was opened at 2007-03-29 23:45 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Roger Upole (rupole) Assigned to: Nobody/Anonymous (nobody) >Summary: Speed up PyArg_ParseTupleAndKeywords() and improve error msg Initial Comment: This is a fix for [ 1283289 ] PyArg_ParseTupleAndKeywords gives misleading error message. It also yields a 10-15% decrease in cpu usage, depending on the number of arguments. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:37 Message: Logged In: YES user_id=33168 Originator: NO The patch to the test code looks fine, although there are some style issues. Some lines over 80 columns and there needs to be spaces around = and other binary operators. I need to review the C code changes, but this patch looks interesting. Roger, in the future, can you make a single patch with all the files. It's easier to download and apply one patch with multiple files than 3 diff patches. ---------------------------------------------------------------------- Comment By: Roger Upole (rupole) Date: 2007-04-04 02:40 Message: Logged In: YES user_id=771074 Originator: YES File Added: test_getargs2.patch ---------------------------------------------------------------------- Comment By: Roger Upole (rupole) Date: 2007-04-03 22:39 Message: Logged In: YES user_id=771074 Originator: YES File Added: _testcapimodule.patch ---------------------------------------------------------------------- Comment By: Roger Upole (rupole) Date: 2007-04-03 22:26 Message: Logged In: YES user_id=771074 Originator: YES File Added: getargs_1.patch ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 11:00 Message: Logged In: YES user_id=1740099 Originator: NO Patched code compiles and passes test suite, and looks cleaner than the code it replaces. It works on the cases the bug report. There should be unit tests added to Lib/test/test_getargs2; I've skimmed the tests in there and can't see any that test combinations of kw, tuple and positional arguments. The patch replaces significant chunks of arg parsing code, so I'd be more convinced of its correctness if there were some unit tests. Certainly there should be at least one test for the bug the patch fixes. The code is 99% written in the prevailing style, but: Source contains several over-long lines, and non-standard comments, ie you should use /* XXX something .. */ and not /* ??? something .. ??? */ Also there are a few minor spacing inconsistencies which would be nice to tidy up. The diff is one level too high up - it contains the directory Python2.5/ It applies cleanly to the trunk though, it just makes it slightly more difficult to apply the patch. The original bug report requests a post to python-dev describing the new error messages. I suggest this post includes all cases where the new code produces a different message than the old. I suggest to proceed: 1. Add unit tests, both for the bug-fix and for existing functionality which has been modified 2. Fix up comments and long lines 3. Check with python-dev about new error messages ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691070&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:44:22 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:44:22 -0700 Subject: [Patches] [ python-Patches-1703286 ] bug fix: ctypes truncates 64-bit pointers Message-ID: Patches item #1703286, was opened at 2007-04-18 18:09 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: bug fix: ctypes truncates 64-bit pointers Initial Comment: At several places _ctypes module converts a python integer to a void* pointer by first extracting an argument into an int. Attached patch changes int to Py_ssize_t that is guaranteed to be wide enough to hold a pointer. ---------------------------------------------------------------------- >Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-19 02:44 Message: Logged In: YES user_id=835142 Originator: YES Here are (failing) test cases: Python 2.5 (r25:51908, Oct 4 2006, 18:25:28) [GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> dlopen('libc.so.6') 182896775816 >>> dlclose(_) Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> dlsym(_, 'printf') Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_function(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_cdeclfunction(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum With the patch: >>> from _ctypes import * >>> dlopen('libc.so.6') 182894214624 >>> dlclose(_) I don't know how to meaningfully exercise call_function and call_cdeclfunction to trigger the bug because in the default (small memory) model on AMD64 function pointers always fit into 32 bits. Maybe a pseudo-function pointer created by an ffi callback can be used for this purpose. I am not sure where these test belong in the testsuit. Maybe in ctypes/test/test_loading.py? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 01:56 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch. The changes look correct, however, can you create a test for these cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 From noreply at sourceforge.net Thu Apr 19 08:49:58 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 18 Apr 2007 23:49:58 -0700 Subject: [Patches] [ python-Patches-1703286 ] bug fix: ctypes truncates 64-bit pointers Message-ID: Patches item #1703286, was opened at 2007-04-18 15:09 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) >Assigned to: Thomas Heller (theller) Summary: bug fix: ctypes truncates 64-bit pointers Initial Comment: At several places _ctypes module converts a python integer to a void* pointer by first extracting an argument into an int. Attached patch changes int to Py_ssize_t that is guaranteed to be wide enough to hold a pointer. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:49 Message: Logged In: YES user_id=33168 Originator: NO Thomas would be the best person to answer you question about where it should go. Maybe he has some other ideas how to test this. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 23:44 Message: Logged In: YES user_id=835142 Originator: YES Here are (failing) test cases: Python 2.5 (r25:51908, Oct 4 2006, 18:25:28) [GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> dlopen('libc.so.6') 182896775816 >>> dlclose(_) Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> dlsym(_, 'printf') Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_function(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_cdeclfunction(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum With the patch: >>> from _ctypes import * >>> dlopen('libc.so.6') 182894214624 >>> dlclose(_) I don't know how to meaningfully exercise call_function and call_cdeclfunction to trigger the bug because in the default (small memory) model on AMD64 function pointers always fit into 32 bits. Maybe a pseudo-function pointer created by an ffi callback can be used for this purpose. I am not sure where these test belong in the testsuit. Maybe in ctypes/test/test_loading.py? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 22:56 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch. The changes look correct, however, can you create a test for these cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 From noreply at sourceforge.net Thu Apr 19 09:12:16 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 00:12:16 -0700 Subject: [Patches] [ python-Patches-1552880 ] Unicode Imports Message-ID: Patches item #1552880, was opened at 2006-09-05 11:11 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1552880&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Kristj?n Valur (krisvale) Assigned to: Nobody/Anonymous (nobody) Summary: Unicode Imports Initial Comment: This patch modifies the import mechanism to fully support unicode pathnames on Windows. It does this by first converting each member of sys.path to utf-8. strings are encoded using the current locale. The whole of the import logic is then unchanged and works on the utf-8 strings as though they were regular ascii strings in the current locale. Only when file operations are done, such as stat() and open(), do we then convert from utf-8 back to unicode and use the Windows unicode APIs for the job. This is also done when initializing Module objects. This approach has the benefit of being of having a low impact on the importing logic, and is thus easy to verify. There is however some overhead with the conversions. At CCP games we used this approach, backported to python 2.3, to get unicode imports working for our game, EVE Online, and thereby solving installation issues in the far east. This patch is submitted as demonstration code to the python community. I would like to see unicode fully supported in 2.6. Cheers, Kristj??n ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 00:12 Message: Logged In: YES user_id=33168 Originator: NO Any function which is not static to a file, must be prefixed with Py or _Py. There are several lines which are over 80 columns and should be wrapped. Why is errno set in open_utf8, etc? Indentation was messed up at least in one place in Objects/moduleobject.c on a DECREF line. I can't provide any guidance on the windows specific code. Where do _wstat and _wfopen come from? There isn't a man page on my Unix box. I'm not sure if the exist in a library anywhere. I didn't see any changes to configure to verify if these exist or not. If Py_UNICODE_IMPORT, does that necessarily mean these APIs exist? (It's possible this code was inside an #if WINDOWS and I couldn't tell from the patch.) ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2007-04-17 03:38 Message: Logged In: YES user_id=1262199 Originator: YES I have uplodaed unicodepatch4.patch, which simplifies this a bit. __file__ and __path__ components are now stored in filesystemencoding if possible. and non-unicode paths are assumed to be in filesystemencoding. This minimizes the impact of the change. File Added: Unicodeimport4.patch ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-09-12 13:17 Message: Logged In: YES user_id=21627 krisvale: indeed, option 4 is platform dependent. Notice that on Linux, the file system encoding won't necessarily be UTF-8. Instead, the value depends on the locale, so it may be latin-1, latin-9, gb2312, ... This makes it even more dependent on the platform, and even the current user being logged in (such is life with locale-based approaches; the same is mostly true for Windows: "mbcs" can mean nearly anything). option 1) is Py3k-safe, where path names will be Unicode strings always. As you say, Unicode is a virulent type, so this approach would need a wide consensus. I'm personally leaning towards option 2: it is nearly backwards compatible, except for obscure cases where people have mbcs-encodable entries in sys.path already, and it is independent of manipulations of the system encoding. I also think that processing of PYTHONPATH should take Unicode into account, i.e. we should use _wgetenv to access PYTHONPATH in 2.6. That would make the feature truly useful, as then people could actually set sys.path to non-mbcs directlories from the outside. Notice that W9x support can be dropped in 2.6, so a W9x-compatible solution won't be required. In any case, I'd like to encourage you to continue working on this issue. I, too, like to see it in 2.6, but I did so ever since 2.1 or so (before PEP 277 was implemented), and it was wishful thinking. Somebody has to take action, and it is likely that it won't one of the past regular contributors (or else they had contributed it long ago - although I think Thomas Heller had something working at one point). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2006-09-12 04:29 Message: Logged In: YES user_id=29957 There's a variety of modules in the standard library that reference __file__ - if it's potentially going to be a unicode string, these are going to need to be checked, as are their callers :-/ (Now that I've looked closer at some of the issues, I'm extremely glad this didn't go into 2.5 final at this late stage) ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2006-09-12 02:38 Message: Logged In: YES user_id=1262199 I submitted this mostly as a demonstration. I don't think the approach is necessarily suitable for a final implementation because of the use of utf-8 as an intermediate representation and the price of the conversions that keep happening. But perhaps this is the way to go, if we consider utf-8 to be a stage-1 default file system encoding for win32. I also agree that 4 is probably the most sensible approach. What about discrepancies between e.g. linux and windows then, when including from a non-trivial path? On linux we would get utf-8, on windows unicode? 1) would actually make a lot of sense, only in my experience this tends to lead to a kind of unicode-hell since a program touched by one unicode object tends to have it percolating down into every corner. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-09-09 05:31 Message: Logged In: YES user_id=21627 First: Do you want to continue to work on this, or do you consider this just "demonstration code" (i.e. not contributed for inclusion in Python), hoping that somebody else implements this feature? I think the behavior of __file__ must be more consistent across platforms, and the selected behaviour must be documented somewhere. Several definitions of "consistent behavior" come to mind: 1. __file__ is always a Unicode string 2. __file__ is a byte string if its ASCII, else Unicode 3. __file__ is a byte string if its in the system encoding, else Unicode 4. __file__ is a byte string if its in the file system encoding, else Unicode. The documentation needs to be updated in several places, e.g. also for inspect.getfile. I would expect that pydoc would also need to be updated. Selecting from the options above: I believe 4 is most compatible with previous versions; 1 and 2 are most convenient to work with in applications like pydoc which have to generate HTML (1 is easier to work with, 2 is more compatible with previous versions). ---------------------------------------------------------------------- Comment By: Kristj?n Valur (krisvale) Date: 2006-09-09 04:38 Message: Logged In: YES user_id=1262199 >From the top of my head, it is now unicode. I consider trying to convert it back to the default encoding but decided not to to keep the patch brief. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-09-08 14:03 Message: Logged In: YES user_id=21627 What is the value of the __file__ attribute of a module when this patch is used? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1552880&group_id=5470 From noreply at sourceforge.net Thu Apr 19 11:41:19 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 02:41:19 -0700 Subject: [Patches] [ python-Patches-1602133 ] non-framework built python fails to define environ properly Message-ID: Patches item #1602133, was opened at 2006-11-24 07:38 Message generated for change (Comment added) made by skiolding You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1602133&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Macintosh Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: paul (ihavetopee) Assigned to: Jack Jansen (jackjansen) Summary: non-framework built python fails to define environ properly Initial Comment: but no one but me seems to build python the unix-y way on darwin/macosx anymore so i guess i'm the only one that cares btw, python without environ defined makes all third party python bindings segfault this has been pissing me off for months only tracked it down yesterday here's my lackluster patch feel free to find something more apropos to conditionalize the diff with and commit it or not i don't care anymore ---------------------------------------------------------------------- Comment By: Adrian Silva (skiolding) Date: 2007-04-19 09:41 Message: Logged In: YES user_id=1771816 Originator: NO Wich MacOsx/python version does this affect? Python from svn trunk (r54849) on a Tiger (10.4.9) seems to define os.environ properly, with and without the patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1602133&group_id=5470 From noreply at sourceforge.net Thu Apr 19 16:52:08 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 07:52:08 -0700 Subject: [Patches] [ python-Patches-1698917 ] ZipFile.printdir fix (2.6) Message-ID: Patches item #1698917, was opened at 2007-04-11 22:27 Message generated for change (Comment added) made by alanmcintyre You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.6) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). Passes full regrtest.py -r ---------------------------------------------------------------------- >Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-19 09:52 Message: Logged In: YES user_id=1115903 Originator: YES I would be glad to; is there a preferred way to test the output of a print statement? If I have time today I can go dig around in the stdlib to look for examples of it, but any pointers would be welcome. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 01:22 Message: Logged In: YES user_id=33168 Originator: NO Alan, thanks for the patch. Would you create a test case for this? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 From noreply at sourceforge.net Thu Apr 19 17:59:30 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 08:59:30 -0700 Subject: [Patches] [ python-Patches-1698917 ] ZipFile.printdir fix (2.6) Message-ID: Patches item #1698917, was opened at 2007-04-11 22:27 Message generated for change (Comment added) made by alanmcintyre You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.6) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). Passes full regrtest.py -r ---------------------------------------------------------------------- >Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-19 10:59 Message: Logged In: YES user_id=1115903 Originator: YES I see that there's already a test that checks printdir (it just doesn't check the modified time). I'll work up a modified version of that to check the situation that caused the original bug. ---------------------------------------------------------------------- Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-19 09:52 Message: Logged In: YES user_id=1115903 Originator: YES I would be glad to; is there a preferred way to test the output of a print statement? If I have time today I can go dig around in the stdlib to look for examples of it, but any pointers would be welcome. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 01:22 Message: Logged In: YES user_id=33168 Originator: NO Alan, thanks for the patch. Would you create a test case for this? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 From noreply at sourceforge.net Thu Apr 19 18:34:56 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 09:34:56 -0700 Subject: [Patches] [ python-Patches-1698917 ] ZipFile.printdir fix (2.6) Message-ID: Patches item #1698917, was opened at 2007-04-11 22:27 Message generated for change (Comment added) made by alanmcintyre You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alan McIntyre (alanmcintyre) Assigned to: Nobody/Anonymous (nobody) Summary: ZipFile.printdir fix (2.6) Initial Comment: This patch for 2.5 is in reference to bug [ 1698398 ] - "wrong % of params for format string in ZipFile.printdir()." It makes the following changes: - Modify ZipFile.printdir() to treat the ZipInfo.date_time object in a way that is similar to other code in zipfile.py. - Modified ZipFile.writestr() to pass a 6-item tuple to the ZipInfo constructor (consistent with other treatment of ZipInfo construction in the zipfile module) instead of the full result from time.localtime(). Passes full regrtest.py -r ---------------------------------------------------------------------- >Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-19 11:34 Message: Logged In: YES user_id=1115903 Originator: YES I added a test for printdir. I also updated some comments to reflect that the modified date is now tested, and changed the imports at the top of the test module to conform to PEP8. File Added: zipfile-printdir-2.diff ---------------------------------------------------------------------- Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-19 10:59 Message: Logged In: YES user_id=1115903 Originator: YES I see that there's already a test that checks printdir (it just doesn't check the modified time). I'll work up a modified version of that to check the situation that caused the original bug. ---------------------------------------------------------------------- Comment By: Alan McIntyre (alanmcintyre) Date: 2007-04-19 09:52 Message: Logged In: YES user_id=1115903 Originator: YES I would be glad to; is there a preferred way to test the output of a print statement? If I have time today I can go dig around in the stdlib to look for examples of it, but any pointers would be welcome. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 01:22 Message: Logged In: YES user_id=33168 Originator: NO Alan, thanks for the patch. Would you create a test case for this? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698917&group_id=5470 From noreply at sourceforge.net Thu Apr 19 21:40:29 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 12:40:29 -0700 Subject: [Patches] [ python-Patches-1602133 ] non-framework built python fails to define environ properly Message-ID: Patches item #1602133, was opened at 2006-11-23 23:38 Message generated for change (Comment added) made by ihavetopee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1602133&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Macintosh Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: paul (ihavetopee) Assigned to: Jack Jansen (jackjansen) Summary: non-framework built python fails to define environ properly Initial Comment: but no one but me seems to build python the unix-y way on darwin/macosx anymore so i guess i'm the only one that cares btw, python without environ defined makes all third party python bindings segfault this has been pissing me off for months only tracked it down yesterday here's my lackluster patch feel free to find something more apropos to conditionalize the diff with and commit it or not i don't care anymore ---------------------------------------------------------------------- >Comment By: paul (ihavetopee) Date: 2007-04-19 12:40 Message: Logged In: YES user_id=1652872 Originator: YES right now, i'm using macosx 10.2/python 2.4 but this should affect any macosx/darwin that's building python as a library instead of a framework meaning the symbol _environ will be completely undefined and no third party python bindings will work from posixmodule.c /* On Darwin/MacOSX a shared library or framework has no access to ** environ directly, we must obtain it with _NSGetEnviron(). */ however, the #ifdef that comment is contained in only takes the framework into consideration ---------------------------------------------------------------------- Comment By: Adrian Silva (skiolding) Date: 2007-04-19 02:41 Message: Logged In: YES user_id=1771816 Originator: NO Wich MacOsx/python version does this affect? Python from svn trunk (r54849) on a Tiger (10.4.9) seems to define os.environ properly, with and without the patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1602133&group_id=5470 From noreply at sourceforge.net Fri Apr 20 00:49:43 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 15:49:43 -0700 Subject: [Patches] [ python-Patches-1703972 ] fixes bug #1703952 - ctypes: problem with large integers Message-ID: Patches item #1703972, was opened at 2007-04-19 18:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703972&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: fixes bug #1703952 - ctypes: problem with large integers Initial Comment: Attached patch adds overflow checks for conversions from python to c integers. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703972&group_id=5470 From noreply at sourceforge.net Fri Apr 20 08:02:35 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 19 Apr 2007 23:02:35 -0700 Subject: [Patches] [ python-Patches-1698768 ] Update to Macintosh library documentation Message-ID: Patches item #1698768, was opened at 2007-04-11 22:40 Message generated for change (Comment added) made by ronaldoussoren You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Kevin Walzer (wordtech) >Assigned to: Brett Cannon (bcannon) Summary: Update to Macintosh library documentation Initial Comment: The attached file updates section 1 of the Macintosh library documentation, which covers getting started--installing, configuring, and learning Python on the Mac. The file updates the section to reflect changes in the Mac version of Python since 2.4 (switching to IDLE as the standard IDE, supporting universal binaries, etc.) and adds sections on application deployment with py2app, GUI toolkits, application scripting, and more. Much of the current documentation in this section is obsolete and therefore inaccurate. The updated documentation is a simple text file: I am not an expert with LaTeX. I understand the Python documentation team can handle the LaTeX formatting. ---------------------------------------------------------------------- >Comment By: Ronald Oussoren (ronaldoussoren) Date: 2007-04-20 08:02 Message: Logged In: YES user_id=580910 Originator: NO This looks great. I'm definitely in favor of using this document instead of the current version. IMHO this change should also be backported to the 2.5 branch of the documention. Brett: I'm assigning this back to you because my LaTeX-fu is seriously rusted. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2007-04-12 01:30 Message: Logged In: YES user_id=357491 Originator: NO Thanks for the work, Kevin! Ronald, can you have a look at this and approve or reject? Once that is done we can work on converting it to LaTeX. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 From noreply at sourceforge.net Fri Apr 20 09:39:47 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 00:39:47 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 03:39 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Fri Apr 20 13:48:29 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 04:48:29 -0700 Subject: [Patches] [ python-Patches-1691032 ] Migrate test_minidom.py to unittest Message-ID: Patches item #1691032, was opened at 2007-03-30 00:15 Message generated for change (Settings changed) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: Migrate test_minidom.py to unittest Initial Comment: Update Lib/test/test_minidom.py to use the unittest module. Also: I noticed the test_minidom.py contains some code (now long dead) to check for reference cycles. The code is active only when Node.allnodes exists; this hasn't existed for over four years now. After re-implementing that test using gc.DEBUG_SAVEALL, I found that many tests failed because of missing unlink() calls. I added them. After that, one test was still failing. This is because expatbuilder.parseString() and friends do not clean up some cyclic references on error. The patch includes a fix for that (in Lib/xml/dom/expatbuilder.py). An alternative would be to ditch the cycle-checking test entirely. I can submit a patch that does that instead, if desired. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-20 07:48 Message: Logged In: YES user_id=18139 Originator: YES There's nothing active here. This should just be Closed Rejected. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 02:29 Message: Logged In: YES user_id=33168 Originator: NO Jason, can you clarify your last comment. Wrt the "change [not] worth making", do you mean this patch or a different change? I just want to know what in this patch is still active and needs review before spending time. Thanks. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-04 22:01 Message: Logged In: YES user_id=18139 Originator: YES Collin: Thanks for taking the time to look at this. I don't think this change is worth making. I've started working on some new features in minidom and it seems like every time I exercise a new piece of code, the gc finds a small number of objects in cycles. For example, parseFragmentString() seems to leave a few objects (even with expatbuilder.second.patch). We're just going to have to live with it-- I don't think anyone has time or desire to hunt down them all down. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-01 17:18 Message: Logged In: YES user_id=18139 Originator: YES File Added: expatbuilder.second.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-01 17:18 Message: Logged In: YES user_id=18139 Originator: YES OK, I've attached two new patches. test_minidom.second.patch - Changes "confirm" to "assert_" or "assertEquals", breaks multi-line assertions into many one-line assertions, adds convenience methods "assertSameNode" and "assertNotSameNode", etc. Also replaces the Node.allnodes test with gc-based test and adds a few calls to unlink(). expatbuilder.second.patch - Fixes the test failure discovered by the new gc testing. File Added: test_minidom.second.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-30 02:49 Message: Logged In: YES user_id=1344176 Originator: NO test_minidom already uses unittest (as of patch #1683397, SVN r54603). Could you update your patch to the latest trunk revision? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1691032&group_id=5470 From noreply at sourceforge.net Fri Apr 20 18:49:22 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 09:49:22 -0700 Subject: [Patches] [ python-Patches-1704473 ] textwrap.py mod. for jython Message-ID: Patches item #1704473, was opened at 2007-04-20 16:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704473&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tim Couper (drtimcouper) Assigned to: Nobody/Anonymous (nobody) Summary: textwrap.py mod. for jython Initial Comment: Extended textwrap.py so that it is operable by jython ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704473&group_id=5470 From noreply at sourceforge.net Fri Apr 20 18:50:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 09:50:54 -0700 Subject: [Patches] [ python-Patches-1704474 ] test_optparse.py mod. for jython Message-ID: Patches item #1704474, was opened at 2007-04-20 16:50 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704474&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tim Couper (drtimcouper) Assigned to: Nobody/Anonymous (nobody) Summary: test_optparse.py mod. for jython Initial Comment: Extension of test code for jython ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704474&group_id=5470 From noreply at sourceforge.net Fri Apr 20 20:48:03 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 11:48:03 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Nobody/Anonymous (nobody) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Fri Apr 20 21:10:49 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 12:10:49 -0700 Subject: [Patches] [ python-Patches-1703972 ] fixes bug #1703952 - ctypes: problem with large integers Message-ID: Patches item #1703972, was opened at 2007-04-20 00:49 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703972&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None >Status: Closed >Resolution: Invalid Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) >Assigned to: Thomas Heller (theller) Summary: fixes bug #1703952 - ctypes: problem with large integers Initial Comment: Attached patch adds overflow checks for conversions from python to c integers. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2007-04-20 21:10 Message: Logged In: YES user_id=11105 Originator: NO I have closed bug #1703952 as invalid, so the same should be done to this patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703972&group_id=5470 From noreply at sourceforge.net Fri Apr 20 21:33:19 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 12:33:19 -0700 Subject: [Patches] [ python-Patches-1703286 ] bug fix: ctypes truncates 64-bit pointers Message-ID: Patches item #1703286, was opened at 2007-04-19 00:09 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Thomas Heller (theller) Summary: bug fix: ctypes truncates 64-bit pointers Initial Comment: At several places _ctypes module converts a python integer to a void* pointer by first extracting an argument into an int. Attached patch changes int to Py_ssize_t that is guaranteed to be wide enough to hold a pointer. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2007-04-20 21:33 Message: Logged In: YES user_id=11105 Originator: NO The patch is only correct for Python2.5 and later, earlier Python versions don't have the Py_ssize_t type (or, when it's defined, it has still the same size as int in these versions). Unfortunately the 'n' typecode also isn't available in these versions. Also the patch has to be extended for Windows, the LoadLibrary function has the same problem. Would you like to extend this patch (for Windows, I can do it myself later)? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 08:49 Message: Logged In: YES user_id=33168 Originator: NO Thomas would be the best person to answer you question about where it should go. Maybe he has some other ideas how to test this. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-19 08:44 Message: Logged In: YES user_id=835142 Originator: YES Here are (failing) test cases: Python 2.5 (r25:51908, Oct 4 2006, 18:25:28) [GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> dlopen('libc.so.6') 182896775816 >>> dlclose(_) Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> dlsym(_, 'printf') Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_function(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_cdeclfunction(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum With the patch: >>> from _ctypes import * >>> dlopen('libc.so.6') 182894214624 >>> dlclose(_) I don't know how to meaningfully exercise call_function and call_cdeclfunction to trigger the bug because in the default (small memory) model on AMD64 function pointers always fit into 32 bits. Maybe a pseudo-function pointer created by an ffi callback can be used for this purpose. I am not sure where these test belong in the testsuit. Maybe in ctypes/test/test_loading.py? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 07:56 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch. The changes look correct, however, can you create a test for these cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 From noreply at sourceforge.net Fri Apr 20 22:04:15 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 13:04:15 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 20:48 Message generated for change (Settings changed) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None >Priority: 6 Private: No Submitted By: Raghuram Devarakonda (draghuram) >Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Fri Apr 20 22:15:36 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 13:15:36 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-20 22:15 Message: Logged In: YES user_id=642936 Originator: NO I checked in a fix in the release25-maint branch (rev. 54889), so it will show up in the 2.5.2 release. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 16:07 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 15:57 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-07 11:33 Message: Logged In: YES user_id=642936 Originator: NO Thank you for making clear that this is a regression. This is a completely different thing and makes the previous discussion obsolete... Unfortunately, this will not go into 2.5.1, we're too late. But I will come up with a reasonable fix and testcases for 2.5.2. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 00:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 23:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Fri Apr 20 22:47:50 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 13:47:50 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 6 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Fri Apr 20 22:51:10 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 13:51:10 -0700 Subject: [Patches] [ python-Patches-1700467 ] stack size of python_d.exe on VC6 Message-ID: Patches item #1700467, was opened at 2007-04-14 00:14 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700467&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Nobody/Anonymous (nobody) Summary: stack size of python_d.exe on VC6 Initial Comment: Hello. I created the patch to expand stack size of python_d.exe(debug build) on VC6 to 1MB automatically. Without this patch, test_exceptions.py (testInfiniteRecursion) causes halt with an error message dialog. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:51 Message: Logged In: YES user_id=18139 Originator: NO +0. But note that 0x1000000 is actually 16MB, not 1MB! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700467&group_id=5470 From noreply at sourceforge.net Fri Apr 20 22:56:26 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 13:56:26 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 6 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- >Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 16:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Fri Apr 20 23:07:07 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 14:07:07 -0700 Subject: [Patches] [ python-Patches-1703286 ] bug fix: ctypes truncates 64-bit pointers Message-ID: Patches item #1703286, was opened at 2007-04-18 18:09 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Thomas Heller (theller) Summary: bug fix: ctypes truncates 64-bit pointers Initial Comment: At several places _ctypes module converts a python integer to a void* pointer by first extracting an argument into an int. Attached patch changes int to Py_ssize_t that is guaranteed to be wide enough to hold a pointer. ---------------------------------------------------------------------- >Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-20 17:07 Message: Logged In: YES user_id=835142 Originator: YES I think you meant FreeLibrary function. LoadLibrary seems ok. I am attaching a new patch that should be pre-2.5 compatible and which extends to FreeLibrary. I don't have access to windows, so please check/test that it works. File Added: _ctypes-callproc-1.patch ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2007-04-20 15:33 Message: Logged In: YES user_id=11105 Originator: NO The patch is only correct for Python2.5 and later, earlier Python versions don't have the Py_ssize_t type (or, when it's defined, it has still the same size as int in these versions). Unfortunately the 'n' typecode also isn't available in these versions. Also the patch has to be extended for Windows, the LoadLibrary function has the same problem. Would you like to extend this patch (for Windows, I can do it myself later)? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 02:49 Message: Logged In: YES user_id=33168 Originator: NO Thomas would be the best person to answer you question about where it should go. Maybe he has some other ideas how to test this. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-19 02:44 Message: Logged In: YES user_id=835142 Originator: YES Here are (failing) test cases: Python 2.5 (r25:51908, Oct 4 2006, 18:25:28) [GCC 3.4.4 20050721 (Red Hat 3.4.4-2)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> dlopen('libc.so.6') 182896775816 >>> dlclose(_) Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> dlsym(_, 'printf') Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_function(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum >>> call_cdeclfunction(_, ()) # pretend that _ is a function pointer Traceback (most recent call last): File "", line 1, in OverflowError: signed integer is greater than maximum With the patch: >>> from _ctypes import * >>> dlopen('libc.so.6') 182894214624 >>> dlclose(_) I don't know how to meaningfully exercise call_function and call_cdeclfunction to trigger the bug because in the default (small memory) model on AMD64 function pointers always fit into 32 bits. Maybe a pseudo-function pointer created by an ffi callback can be used for this purpose. I am not sure where these test belong in the testsuit. Maybe in ctypes/test/test_loading.py? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 01:56 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch. The changes look correct, however, can you create a test for these cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703286&group_id=5470 From noreply at sourceforge.net Fri Apr 20 23:13:21 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 14:13:21 -0700 Subject: [Patches] [ python-Patches-1678345 ] A fix for the bug #1528074 [warning: quite slow] Message-ID: Patches item #1678345, was opened at 2007-03-11 11:28 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1678345&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Denys Rtveliashvili (rtvd) Assigned to: Nobody/Anonymous (nobody) Summary: A fix for the bug #1528074 [warning: quite slow] Initial Comment: This is a fix for the bug #1528074 in the difflib's SequenceMatcher which makes (among other possible things) find_longest_match return invalid results sometimes. The previously submitted test case for this bug has the #1678339 ID. The find_longest_match and __chain_b methods in the SequenceMatcher are perfectly optimized. The find_longest_match would work both fast and correctly if only the __chain_b did not break it's assumptions. The find_longest_match assumes that the b2j mapping has a mapping of all non-junk elements in b to lists of their indices in the "b" list. However, when __chain_b creates the b2j mapping, it removes popular elements from the list and marking the elements as popular in the "populardict". As a result, the find_longest_match method can't find the indices for the popular elements and they become automatically considered as something like a junk. I tried to fix the bug by both changing the find_longest_match and __chain_b methods. No matter how hard I tried, the change dropped the performance and slowed down the matching by 5-10 times. The impact of find_longest_match method was larger, so I decided to send a patch for the __chain_b. Please, note, that even though the method starts to work properly and the test cases pass on my computer just fine, the ingenious optimizations performed before become broken, so it would be great if a guru in Python code optimization tries to improve the things a bit. One more point: if the indices are not removed, the memory consumption on the large strings can become quite great. If this is a serious concern, a fix in the find_longest_match will need to be done instead. However that fix would probably be far less efficient that this one. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 17:13 Message: Logged In: YES user_id=18139 Originator: NO Recommend closing this. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-03-18 21:59 Message: Logged In: YES user_id=764593 Originator: NO I think the bug is documentation only. According to the comments (but not the docstring) find_longest_match returns the longest _interesting_ match. A single element appearing too often is likely to cause spurious matches, and is therefore not interesting. I do agree that this should be noted more prominently, so people don't try things like comparing text strings letter-by-letter (where 1% is far too low a threshhold for a 26-character alphabet). And yes, the comments on popular are correct -- it ignores elements which constitute *more* than 1%. I recommend closing this set of tracker items. If you could submit changes to the documentation (docstrings and/or help files; maybe even the comments), I would recommend applying them. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1678345&group_id=5470 From noreply at sourceforge.net Fri Apr 20 23:45:07 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 14:45:07 -0700 Subject: [Patches] [ python-Patches-1673007 ] urllib2 requests history + HEAD support Message-ID: Patches item #1673007, was opened at 2007-03-03 09:01 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1673007&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: KDanilov aka koder (koder_ua) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2 requests history + HEAD support Initial Comment: 1)Add history off all sent and received headers/requests to addinfourl object. Save redirections history too. >>> fd = urllib2.urlopen("http://www.python.org/") >>> print fd.history[0].request_line GET / HTTP/1.1 >>> print fd.history[0].sended [('Accept-Encoding', 'identity'), ('Host', 'www.python.org'), ('Connection', 'cl ose'), ('User-Agent', 'Python-urllib/2.6')] 2)Add support for HEAD (and other) requests: >>> fd = urllib2.urlopen("http://www.python.org/", request_cmd = "HEAD") >>> print len(fd.read()) 0 Please send email here: koder_dot_mail_at_gmail_dot_com ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 17:45 Message: Logged In: YES user_id=18139 Originator: NO koder_ua, what's the history useful for? I'm not against it... exactly... it just seems like I would never use it. If I did want a history of HTTP activity, I would need to maintain it myself. I wouldn't want it tied to a particular object--these urlopen() objects, typically you open them, read them, and then throw them away. I'm iffy on the API for 2) as well, but I can see the appeal. The thing is... this patch is really, really rough. In one of the tests, an attribute is called "sended_hdrs" in one place and "sended_headers" in another. (Actually it should be called "request_headers". Likewise "recived" and "res" should each be spelled "response".) The whitespace isn't PEP 8. It's also missing a doc patch. ---------------------------------------------------------------------- Comment By: KDanilov aka koder (koder_ua) Date: 2007-03-07 04:41 Message: Logged In: YES user_id=1620221 Originator: YES Make diff file with svn and delete previos File Added: http_history_and_rtype_svn.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1673007&group_id=5470 From noreply at sourceforge.net Sat Apr 21 00:17:46 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 15:17:46 -0700 Subject: [Patches] [ python-Patches-1704621 ] interpreter crash when multiplying large lists Message-ID: Patches item #1704621, was opened at 2007-04-20 17:17 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Daniel Stutzbach (agthorr) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when multiplying large lists Initial Comment: Here's a succinct summary of the problem: >>> x = [0] * 2**20 >>> x *= 2**20 Segmentation fault (core dumped) >>> x = [0] * 2**20 >>> x * 2**20 [] >>> The problem is that list_repeat()'s check for an overflow is flawed, and list_inplace_repeat() doesn't check at all. Attached is a patch that adds a correct check to both functions. With the patch, both variations throw a MemoryError exception. The patch is against Python 2.5, but should also fix 2.5.1/2.6/3.0. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 From noreply at sourceforge.net Sat Apr 21 00:19:16 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 15:19:16 -0700 Subject: [Patches] [ python-Patches-1665292 ] Datetime enhancements Message-ID: Patches item #1665292, was opened at 2007-02-21 09:55 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1665292&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Christian Heimes (tiran) Assigned to: Nobody/Anonymous (nobody) Summary: Datetime enhancements Initial Comment: I'm proposing some small enhancements to the datetime module: Add a totimestamp() method to datetime.datetime that returns the seconds since 1/1/1970 00:00:00. The datetime class has already a fromtimestamp() factory but is missing a totimestamp() method. Add a __int__() and __float__() method to datetime.timedelta which return the seconds (seconds + 86400 * days) as int and seconds + miliseconds as float. It would save some typing if somebody needs an integer representation of a timedelta object :] The datetime module is implemented in C. I've never written a Python C extension so I can't help with a patch. Thx ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 18:19 Message: Logged In: YES user_id=18139 Originator: NO tiran, having timedelta objects comparable with numbers is going a bit too far. You end up with: timedelta(0, 1) == 1 but hash(timedelta(0, 1)) != hash(1) This breaks dictionaries. ---------------------------------------------------------------------- Comment By: Christian Heimes (tiran) Date: 2007-03-22 22:38 Message: Logged In: YES user_id=560817 Originator: YES Here is a new patch. I've updated the documentation and enhanced the compare function to compare with ints, longs and floats. This part of the patch should be complete. I'm going to submit a different patch for the other features later. Shall I start a new tracker entry? File Added: timedelta_intfloat.diff ---------------------------------------------------------------------- Comment By: Johann C. Rocholl (jcrocholl) Date: 2007-03-09 11:33 Message: Logged In: YES user_id=656137 Originator: NO The patch confuses microseconds with milliseconds. The divisor in delta_float should be 1000000.0, not 1000.0. The result of the last test should be 86401.000001, not 86401.001. ---------------------------------------------------------------------- Comment By: Christian Heimes (tiran) Date: 2007-02-21 11:16 Message: Logged In: YES user_id=560817 Originator: YES File Added: timedelta.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1665292&group_id=5470 From noreply at sourceforge.net Sat Apr 21 00:37:47 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 15:37:47 -0700 Subject: [Patches] [ python-Patches-1652328 ] stream writing support in wave.py Message-ID: Patches item #1652328, was opened at 2007-02-05 06:45 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1652328&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: tom (tom375) Assigned to: Nobody/Anonymous (nobody) Summary: stream writing support in wave.py Initial Comment: allows wave files to be written to non-seekable file descriptors ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 18:37 Message: Logged In: YES user_id=18139 Originator: NO Paul, if you call writeframesraw() exactly once, this would work. But yes, the patch squelches more errors than it needs to. Definitely should be rejected. ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-03-17 12:23 Message: Logged In: YES user_id=1740099 Originator: NO The code ignores exceptions thrown during seeks and tells, skipping operations in the process. I can't see how this makes the written wave file correct. I would say this patch should be rejected. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1652328&group_id=5470 From noreply at sourceforge.net Sat Apr 21 00:41:44 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 15:41:44 -0700 Subject: [Patches] [ python-Patches-1704621 ] interpreter crash when multiplying large lists Message-ID: Patches item #1704621, was opened at 2007-04-20 18:17 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Daniel Stutzbach (agthorr) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when multiplying large lists Initial Comment: Here's a succinct summary of the problem: >>> x = [0] * 2**20 >>> x *= 2**20 Segmentation fault (core dumped) >>> x = [0] * 2**20 >>> x * 2**20 [] >>> The problem is that list_repeat()'s check for an overflow is flawed, and list_inplace_repeat() doesn't check at all. Attached is a patch that adds a correct check to both functions. With the patch, both variations throw a MemoryError exception. The patch is against Python 2.5, but should also fix 2.5.1/2.6/3.0. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 18:41 Message: Logged In: YES user_id=18139 Originator: NO Yes, I see it. Patch looks good. I haven't tested it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 From noreply at sourceforge.net Sat Apr 21 05:04:49 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 20:04:49 -0700 Subject: [Patches] [ python-Patches-1704621 ] interpreter crash when multiplying large lists Message-ID: Patches item #1704621, was opened at 2007-04-20 15:17 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Daniel Stutzbach (agthorr) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when multiplying large lists Initial Comment: Here's a succinct summary of the problem: >>> x = [0] * 2**20 >>> x *= 2**20 Segmentation fault (core dumped) >>> x = [0] * 2**20 >>> x * 2**20 [] >>> The problem is that list_repeat()'s check for an overflow is flawed, and list_inplace_repeat() doesn't check at all. Attached is a patch that adds a correct check to both functions. With the patch, both variations throw a MemoryError exception. The patch is against Python 2.5, but should also fix 2.5.1/2.6/3.0. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-20 20:04 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! Can you add a test case for both conditions? ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 15:41 Message: Logged In: YES user_id=18139 Originator: NO Yes, I see it. Patch looks good. I haven't tested it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 From noreply at sourceforge.net Sat Apr 21 05:54:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 20:54:54 -0700 Subject: [Patches] [ python-Patches-1652328 ] stream writing support in wave.py Message-ID: Patches item #1652328, was opened at 2007-02-05 03:45 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1652328&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: tom (tom375) >Assigned to: Neal Norwitz (nnorwitz) Summary: stream writing support in wave.py Initial Comment: allows wave files to be written to non-seekable file descriptors ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-20 20:54 Message: Logged In: YES user_id=33168 Originator: NO Taking advice and reject. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 15:37 Message: Logged In: YES user_id=18139 Originator: NO Paul, if you call writeframesraw() exactly once, this would work. But yes, the patch squelches more errors than it needs to. Definitely should be rejected. ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-03-17 09:23 Message: Logged In: YES user_id=1740099 Originator: NO The code ignores exceptions thrown during seeks and tells, skipping operations in the process. I can't see how this makes the written wave file correct. I would say this patch should be rejected. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1652328&group_id=5470 From noreply at sourceforge.net Sat Apr 21 05:55:32 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 20:55:32 -0700 Subject: [Patches] [ python-Patches-1704621 ] interpreter crash when multiplying large lists Message-ID: Patches item #1704621, was opened at 2007-04-20 17:17 Message generated for change (Comment added) made by agthorr You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Daniel Stutzbach (agthorr) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when multiplying large lists Initial Comment: Here's a succinct summary of the problem: >>> x = [0] * 2**20 >>> x *= 2**20 Segmentation fault (core dumped) >>> x = [0] * 2**20 >>> x * 2**20 [] >>> The problem is that list_repeat()'s check for an overflow is flawed, and list_inplace_repeat() doesn't check at all. Attached is a patch that adds a correct check to both functions. With the patch, both variations throw a MemoryError exception. The patch is against Python 2.5, but should also fix 2.5.1/2.6/3.0. ---------------------------------------------------------------------- >Comment By: Daniel Stutzbach (agthorr) Date: 2007-04-20 22:55 Message: Logged In: YES user_id=6324 Originator: YES Sort of. Below is a test for 32-bit architectures. Unfortunately, on architectures where Py_ssize_t is a 64-bit integer, the multiplication won't overflow and the test really will try to build a list with 2**32 items (which most likely will eventually raise a MemoryError, but it will take a LONG time!). This is probably undesirable. Is there a way to skip this test for non-32-bit architectures? One way would be to only perform the test if sys.maxint <= 2**31. Below is a test that can be added to seq_tests.py: def test_bigrepeat(self): x = self.type2test([0]) x *= 2**16 self.assertRaises(MemoryError, x.__mul__, 2**16) self.assertRaises(MemoryError, x.__imul__, 2**16) While writing the test I found a bug in my patch for the case where the list is already size 0. New patch attached. File Added: list_repeat.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-20 22:04 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! Can you add a test case for both conditions? ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 17:41 Message: Logged In: YES user_id=18139 Originator: NO Yes, I see it. Patch looks good. I haven't tested it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 From noreply at sourceforge.net Sat Apr 21 06:14:51 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 21:14:51 -0700 Subject: [Patches] [ python-Patches-1704621 ] interpreter crash when multiplying large lists Message-ID: Patches item #1704621, was opened at 2007-04-20 15:17 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Daniel Stutzbach (agthorr) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when multiplying large lists Initial Comment: Here's a succinct summary of the problem: >>> x = [0] * 2**20 >>> x *= 2**20 Segmentation fault (core dumped) >>> x = [0] * 2**20 >>> x * 2**20 [] >>> The problem is that list_repeat()'s check for an overflow is flawed, and list_inplace_repeat() doesn't check at all. Attached is a patch that adds a correct check to both functions. With the patch, both variations throw a MemoryError exception. The patch is against Python 2.5, but should also fix 2.5.1/2.6/3.0. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-20 21:14 Message: Logged In: YES user_id=33168 Originator: NO You could always check the value of sys.maxint. The test_bigrepeat below fails quickly on a 64-bit box if you don't have enough memory. :-) If you calculated 16 in the __mul__ calls based on sys.maxint, wouldn't that do what you want? Perhaps something like: reasonable_exp = 16 x *= 2 ** reasonable_exp overflow_exp = int(math.log(sys.maxint, 2)) + 1 - reasonable_exp self.assertRaises(MemoryError, x.__mul__, 2**overflow_exp) ---------------------------------------------------------------------- Comment By: Daniel Stutzbach (agthorr) Date: 2007-04-20 20:55 Message: Logged In: YES user_id=6324 Originator: YES Sort of. Below is a test for 32-bit architectures. Unfortunately, on architectures where Py_ssize_t is a 64-bit integer, the multiplication won't overflow and the test really will try to build a list with 2**32 items (which most likely will eventually raise a MemoryError, but it will take a LONG time!). This is probably undesirable. Is there a way to skip this test for non-32-bit architectures? One way would be to only perform the test if sys.maxint <= 2**31. Below is a test that can be added to seq_tests.py: def test_bigrepeat(self): x = self.type2test([0]) x *= 2**16 self.assertRaises(MemoryError, x.__mul__, 2**16) self.assertRaises(MemoryError, x.__imul__, 2**16) While writing the test I found a bug in my patch for the case where the list is already size 0. New patch attached. File Added: list_repeat.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-20 20:04 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! Can you add a test case for both conditions? ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 15:41 Message: Logged In: YES user_id=18139 Originator: NO Yes, I see it. Patch looks good. I haven't tested it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 From noreply at sourceforge.net Sat Apr 21 08:34:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 23:34:12 -0700 Subject: [Patches] [ python-Patches-1678345 ] A fix for the bug #1528074 [warning: quite slow] Message-ID: Patches item #1678345, was opened at 2007-03-11 18:28 Message generated for change (Comment added) made by rtvd You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1678345&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Denys Rtveliashvili (rtvd) Assigned to: Nobody/Anonymous (nobody) Summary: A fix for the bug #1528074 [warning: quite slow] Initial Comment: This is a fix for the bug #1528074 in the difflib's SequenceMatcher which makes (among other possible things) find_longest_match return invalid results sometimes. The previously submitted test case for this bug has the #1678339 ID. The find_longest_match and __chain_b methods in the SequenceMatcher are perfectly optimized. The find_longest_match would work both fast and correctly if only the __chain_b did not break it's assumptions. The find_longest_match assumes that the b2j mapping has a mapping of all non-junk elements in b to lists of their indices in the "b" list. However, when __chain_b creates the b2j mapping, it removes popular elements from the list and marking the elements as popular in the "populardict". As a result, the find_longest_match method can't find the indices for the popular elements and they become automatically considered as something like a junk. I tried to fix the bug by both changing the find_longest_match and __chain_b methods. No matter how hard I tried, the change dropped the performance and slowed down the matching by 5-10 times. The impact of find_longest_match method was larger, so I decided to send a patch for the __chain_b. Please, note, that even though the method starts to work properly and the test cases pass on my computer just fine, the ingenious optimizations performed before become broken, so it would be great if a guru in Python code optimization tries to improve the things a bit. One more point: if the indices are not removed, the memory consumption on the large strings can become quite great. If this is a serious concern, a fix in the find_longest_match will need to be done instead. However that fix would probably be far less efficient that this one. ---------------------------------------------------------------------- >Comment By: Denys Rtveliashvili (rtvd) Date: 2007-04-21 10:34 Message: Logged In: YES user_id=1416496 Originator: YES I am sure this is NOT a bug in the documentation. There is no such thing as "interesting" match unless it can be described mathematically. Instead, the documentation specifies _precisely_ what exactly the function must do in a very precise manner. The problem is that it does a different thing. Another issue of this algorithm is that it uses a quite slow algorithm. It's computational complexity is O(n*m). Another one based on generalized suffix trees has a much more modest complexity O(n+m). The details are here: http://en.wikipedia.org/wiki/Longest-common_substring_problem ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-21 01:13 Message: Logged In: YES user_id=18139 Originator: NO Recommend closing this. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-03-19 04:59 Message: Logged In: YES user_id=764593 Originator: NO I think the bug is documentation only. According to the comments (but not the docstring) find_longest_match returns the longest _interesting_ match. A single element appearing too often is likely to cause spurious matches, and is therefore not interesting. I do agree that this should be noted more prominently, so people don't try things like comparing text strings letter-by-letter (where 1% is far too low a threshhold for a 26-character alphabet). And yes, the comments on popular are correct -- it ignores elements which constitute *more* than 1%. I recommend closing this set of tracker items. If you could submit changes to the documentation (docstrings and/or help files; maybe even the comments), I would recommend applying them. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1678345&group_id=5470 From noreply at sourceforge.net Sat Apr 21 08:52:38 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 20 Apr 2007 23:52:38 -0700 Subject: [Patches] [ python-Patches-1700467 ] stack size of python_d.exe on VC6 Message-ID: Patches item #1700467, was opened at 2007-04-14 13:14 Message generated for change (Comment added) made by ocean-city You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700467&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Nobody/Anonymous (nobody) Summary: stack size of python_d.exe on VC6 Initial Comment: Hello. I created the patch to expand stack size of python_d.exe(debug build) on VC6 to 1MB automatically. Without this patch, test_exceptions.py (testInfiniteRecursion) causes halt with an error message dialog. ---------------------------------------------------------------------- >Comment By: Hirokazu Yamamoto (ocean-city) Date: 2007-04-21 15:52 Message: Logged In: YES user_id=1200846 Originator: YES Ooops, sorry. I re-uploaded the patch. (1MB was not enough for debug build, so this patch increases stack size to 2MB) File Added: vc6-trunk.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-21 05:51 Message: Logged In: YES user_id=18139 Originator: NO +0. But note that 0x1000000 is actually 16MB, not 1MB! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1700467&group_id=5470 From noreply at sourceforge.net Sat Apr 21 15:29:10 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 21 Apr 2007 06:29:10 -0700 Subject: [Patches] [ python-Patches-1704621 ] interpreter crash when multiplying large lists Message-ID: Patches item #1704621, was opened at 2007-04-20 17:17 Message generated for change (Comment added) made by agthorr You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Daniel Stutzbach (agthorr) Assigned to: Nobody/Anonymous (nobody) Summary: interpreter crash when multiplying large lists Initial Comment: Here's a succinct summary of the problem: >>> x = [0] * 2**20 >>> x *= 2**20 Segmentation fault (core dumped) >>> x = [0] * 2**20 >>> x * 2**20 [] >>> The problem is that list_repeat()'s check for an overflow is flawed, and list_inplace_repeat() doesn't check at all. Attached is a patch that adds a correct check to both functions. With the patch, both variations throw a MemoryError exception. The patch is against Python 2.5, but should also fix 2.5.1/2.6/3.0. ---------------------------------------------------------------------- >Comment By: Daniel Stutzbach (agthorr) Date: 2007-04-21 08:29 Message: Logged In: YES user_id=6324 Originator: YES Yes, I realized the same thing while staring off into space before getting out of bed this morning. :-) Here's a simpler version: x = [0,0,0,0] self.assertRaises(MemoryError, x.__mul__, sys.maxint/2+1) self.assertRaises(MemoryError, x.__imul__, sys.maxint/2+1) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-20 23:14 Message: Logged In: YES user_id=33168 Originator: NO You could always check the value of sys.maxint. The test_bigrepeat below fails quickly on a 64-bit box if you don't have enough memory. :-) If you calculated 16 in the __mul__ calls based on sys.maxint, wouldn't that do what you want? Perhaps something like: reasonable_exp = 16 x *= 2 ** reasonable_exp overflow_exp = int(math.log(sys.maxint, 2)) + 1 - reasonable_exp self.assertRaises(MemoryError, x.__mul__, 2**overflow_exp) ---------------------------------------------------------------------- Comment By: Daniel Stutzbach (agthorr) Date: 2007-04-20 22:55 Message: Logged In: YES user_id=6324 Originator: YES Sort of. Below is a test for 32-bit architectures. Unfortunately, on architectures where Py_ssize_t is a 64-bit integer, the multiplication won't overflow and the test really will try to build a list with 2**32 items (which most likely will eventually raise a MemoryError, but it will take a LONG time!). This is probably undesirable. Is there a way to skip this test for non-32-bit architectures? One way would be to only perform the test if sys.maxint <= 2**31. Below is a test that can be added to seq_tests.py: def test_bigrepeat(self): x = self.type2test([0]) x *= 2**16 self.assertRaises(MemoryError, x.__mul__, 2**16) self.assertRaises(MemoryError, x.__imul__, 2**16) While writing the test I found a bug in my patch for the case where the list is already size 0. New patch attached. File Added: list_repeat.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-20 22:04 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! Can you add a test case for both conditions? ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 17:41 Message: Logged In: YES user_id=18139 Originator: NO Yes, I see it. Patch looks good. I haven't tested it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704621&group_id=5470 From noreply at sourceforge.net Sat Apr 21 17:48:44 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 21 Apr 2007 08:48:44 -0700 Subject: [Patches] [ python-Patches-1675363 ] PEP 3114 -- next() -> __next__() Message-ID: Patches item #1675363, was opened at 2007-03-07 00:21 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1675363&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 >Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: PEP 3114 -- next() -> __next__() Initial Comment: This patch implements PEP 3114. It renames all next() methods to __next__, introduces the new builtin next() with one or two arguments and updates the documentation accordingly. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-21 15:48 Message: Logged In: YES user_id=849994 Originator: YES Done in rev. 54910. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-18 21:54 Message: Logged In: YES user_id=6380 Originator: NO Can you bring this up to date and commit it? ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-07 11:27 Message: Logged In: YES user_id=849994 Originator: YES Version 2 adds tests for the new builtin next(). File Added: __next__.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1675363&group_id=5470 From noreply at sourceforge.net Mon Apr 23 00:15:00 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 22 Apr 2007 15:15:00 -0700 Subject: [Patches] [ python-Patches-1704473 ] textwrap.py mod. for jython Message-ID: Patches item #1704473, was opened at 2007-04-20 11:49 Message generated for change (Comment added) made by cgroves You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704473&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tim Couper (drtimcouper) Assigned to: Nobody/Anonymous (nobody) Summary: textwrap.py mod. for jython Initial Comment: Extended textwrap.py so that it is operable by jython ---------------------------------------------------------------------- Comment By: Charles Groves (cgroves) Date: 2007-04-22 17:15 Message: Logged In: YES user_id=1174327 Originator: NO This patch works around the lack of translate on unicode objects in Jython. This was just fixed in Jython so the patch doesn't need to be applied. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704473&group_id=5470 From noreply at sourceforge.net Mon Apr 23 00:18:47 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 22 Apr 2007 15:18:47 -0700 Subject: [Patches] [ python-Patches-1704473 ] textwrap.py mod. for jython Message-ID: Patches item #1704473, was opened at 2007-04-20 16:49 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704473&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 >Status: Closed >Resolution: Out of Date Priority: 5 Private: No Submitted By: Tim Couper (drtimcouper) Assigned to: Nobody/Anonymous (nobody) Summary: textwrap.py mod. for jython Initial Comment: Extended textwrap.py so that it is operable by jython ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-22 22:18 Message: Logged In: YES user_id=849994 Originator: NO Thanks, closing then. ---------------------------------------------------------------------- Comment By: Charles Groves (cgroves) Date: 2007-04-22 22:15 Message: Logged In: YES user_id=1174327 Originator: NO This patch works around the lack of translate on unicode objects in Jython. This was just fixed in Jython so the patch doesn't need to be applied. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704473&group_id=5470 From noreply at sourceforge.net Mon Apr 23 08:15:09 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 22 Apr 2007 23:15:09 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 22:23 Message generated for change (Settings changed) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: Library (Lib) >Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) >Assigned to: Guido van Rossum (gvanrossum) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-22 23:15 Message: Logged In: YES user_id=33168 Originator: NO I'm not sure why all the code was in there to begin with it was ancient history. Hopefully Guido remembers. Guido, do you see any particular issue with this approach (ie, removing all the gyrations in warnings and using f_code.co_filename? There is a difference in behaviour. If a module modifies its __file__, the old code would pick up the modified version. This new code will find the real file. I'm not sure if that's a bug or a feature though. :-) Similarly with __name__. Adam, I like adding the warning_guard using a with statement. Although I don't think comment should be completely removed. Could you add a comment about why the guard is necessary (ie, the repeated calls/-R part)? Have you tested with -R and with -u all? ie: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: and ./python -E -tt ./Lib/test/regrtest.py -u all (two separate runs). In the last test case you are replacing (the old spam7), why does "sys" become "no context"? ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-15 18:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 14:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 00:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Mon Apr 23 14:06:43 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 05:06:43 -0700 Subject: [Patches] [ python-Patches-1678345 ] A fix for the bug #1528074 [warning: quite slow] Message-ID: Patches item #1678345, was opened at 2007-03-11 11:28 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1678345&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Denys Rtveliashvili (rtvd) Assigned to: Nobody/Anonymous (nobody) Summary: A fix for the bug #1528074 [warning: quite slow] Initial Comment: This is a fix for the bug #1528074 in the difflib's SequenceMatcher which makes (among other possible things) find_longest_match return invalid results sometimes. The previously submitted test case for this bug has the #1678339 ID. The find_longest_match and __chain_b methods in the SequenceMatcher are perfectly optimized. The find_longest_match would work both fast and correctly if only the __chain_b did not break it's assumptions. The find_longest_match assumes that the b2j mapping has a mapping of all non-junk elements in b to lists of their indices in the "b" list. However, when __chain_b creates the b2j mapping, it removes popular elements from the list and marking the elements as popular in the "populardict". As a result, the find_longest_match method can't find the indices for the popular elements and they become automatically considered as something like a junk. I tried to fix the bug by both changing the find_longest_match and __chain_b methods. No matter how hard I tried, the change dropped the performance and slowed down the matching by 5-10 times. The impact of find_longest_match method was larger, so I decided to send a patch for the __chain_b. Please, note, that even though the method starts to work properly and the test cases pass on my computer just fine, the ingenious optimizations performed before become broken, so it would be great if a guru in Python code optimization tries to improve the things a bit. One more point: if the indices are not removed, the memory consumption on the large strings can become quite great. If this is a serious concern, a fix in the find_longest_match will need to be done instead. However that fix would probably be far less efficient that this one. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 08:06 Message: Logged In: YES user_id=18139 Originator: NO rtvd, Thanks for taking the time to post this patch and participate in the discussion. This patch changes intentional, by-design behavior. It would hurt difflib's performance and reduce its usability, by making the matches worse. Subjectively worse, admittedly, but with many programmers sharing similar subjective views. Read the comments in the source code and you'll get the gist. And even a mathematician would dispute your assertion that there's no such thing as "interesting". That's a pretty extreme view. ---------------------------------------------------------------------- Comment By: Denys Rtveliashvili (rtvd) Date: 2007-04-21 02:34 Message: Logged In: YES user_id=1416496 Originator: YES I am sure this is NOT a bug in the documentation. There is no such thing as "interesting" match unless it can be described mathematically. Instead, the documentation specifies _precisely_ what exactly the function must do in a very precise manner. The problem is that it does a different thing. Another issue of this algorithm is that it uses a quite slow algorithm. It's computational complexity is O(n*m). Another one based on generalized suffix trees has a much more modest complexity O(n+m). The details are here: http://en.wikipedia.org/wiki/Longest-common_substring_problem ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 17:13 Message: Logged In: YES user_id=18139 Originator: NO Recommend closing this. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-03-18 21:59 Message: Logged In: YES user_id=764593 Originator: NO I think the bug is documentation only. According to the comments (but not the docstring) find_longest_match returns the longest _interesting_ match. A single element appearing too often is likely to cause spurious matches, and is therefore not interesting. I do agree that this should be noted more prominently, so people don't try things like comparing text strings letter-by-letter (where 1% is far too low a threshhold for a 26-character alphabet). And yes, the comments on popular are correct -- it ignores elements which constitute *more* than 1%. I recommend closing this set of tracker items. If you could submit changes to the documentation (docstrings and/or help files; maybe even the comments), I would recommend applying them. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1678345&group_id=5470 From noreply at sourceforge.net Mon Apr 23 15:40:24 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 06:40:24 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 03:39 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-23 09:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Mon Apr 23 15:53:32 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 06:53:32 -0700 Subject: [Patches] [ python-Patches-1669481 ] subprocess: Support close_fds on Win32 Message-ID: Patches item #1669481, was opened at 2007-02-26 16:47 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1669481&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jon Foster (jongfoster) Assigned to: Nobody/Anonymous (nobody) Summary: subprocess: Support close_fds on Win32 Initial Comment: There is an existing close_fds parameter to the subprocess.Popen() constructor that prevents FDs being copied to the child process on UNIX. This patch makes that parameter also work on Windows (with a caveat). Setting this parameter on Windows was explicitly prohibited by the code, because it prevents redirecting stdin/stdout/stderr. However, the check is too restrictive; it is perfectly reasonable to set this flag if you are not redirecting anything. That is what this patch does. The behaviour is unchanged if you are on Windows, set this flag, and simultaneously try to redirect stdin/stdout/stderr: the call will fail with an exception. This patch is against Python SVN trunk. Kind regards, Jon Foster ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 09:53 Message: Logged In: YES user_id=18139 Originator: NO Patch looks good. +1 (but I haven't tested it either). ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-08 15:28 Message: Logged In: YES user_id=849994 Originator: NO The patch looks good; unfortunately I don't have a Windows to test it, otherwise I'd apply it. ---------------------------------------------------------------------- Comment By: Jon Foster (jongfoster) Date: 2007-03-07 18:11 Message: Logged In: YES user_id=199289 Originator: YES I've been reading the general discussion on python-dev about patches needing documentation and tests. So here's an updated patch that includes docs and tests. File Added: subprocess_win32_close_handles_2.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1669481&group_id=5470 From noreply at sourceforge.net Mon Apr 23 19:21:41 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 10:21:41 -0700 Subject: [Patches] [ python-Patches-1706039 ] Added clearerr() to clear EOF state Message-ID: Patches item #1706039, was opened at 2007-04-23 17:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: jos (josm) Assigned to: Nobody/Anonymous (nobody) Summary: Added clearerr() to clear EOF state Initial Comment: This patch is to fix bug 1523853. Now file.read() works as written in the doc, which says "An empty string is returned when EOF is encountered immediately." Before this fix, An empty string was returned even when the last read() encountered EOF. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 From noreply at sourceforge.net Mon Apr 23 19:31:11 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 10:31:11 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-02 01:23 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) >Assigned to: Neal Norwitz (nnorwitz) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-23 13:31 Message: Logged In: YES user_id=6380 Originator: NO The main reason for using __file__ instead of the filename baked into the code object is that sometimes .pyc files are moved after being compiled; the code object is not updated, but __file__ has the correct path as it reflects how the module was found during import. Also, code objects may have relative pathnames. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 02:15 Message: Logged In: YES user_id=33168 Originator: NO I'm not sure why all the code was in there to begin with it was ancient history. Hopefully Guido remembers. Guido, do you see any particular issue with this approach (ie, removing all the gyrations in warnings and using f_code.co_filename? There is a difference in behaviour. If a module modifies its __file__, the old code would pick up the modified version. This new code will find the real file. I'm not sure if that's a bug or a feature though. :-) Similarly with __name__. Adam, I like adding the warning_guard using a with statement. Although I don't think comment should be completely removed. Could you add a comment about why the guard is necessary (ie, the repeated calls/-R part)? Have you tested with -R and with -u all? ie: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: and ./python -E -tt ./Lib/test/regrtest.py -u all (two separate runs). In the last test case you are replacing (the old spam7), why does "sys" become "no context"? ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-15 21:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 17:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 03:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Mon Apr 23 22:22:10 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 13:22:10 -0700 Subject: [Patches] [ python-Patches-1706039 ] Added clearerr() to clear EOF state Message-ID: Patches item #1706039, was opened at 2007-04-23 13:21 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: jos (josm) Assigned to: Nobody/Anonymous (nobody) Summary: Added clearerr() to clear EOF state Initial Comment: This patch is to fix bug 1523853. Now file.read() works as written in the doc, which says "An empty string is returned when EOF is encountered immediately." Before this fix, An empty string was returned even when the last read() encountered EOF. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-23 16:22 Message: Logged In: YES user_id=835142 Originator: NO I was not able to reproduce the problem. I tried to read through the end of the file, append to the file from an external process and read again. It worked as expected: more data was read. The proposed patch seems to be redundant: once ferror(f->f_fp) is known to be 0, clearerr will do nothing and in the alternative branch clearerr is already there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 From noreply at sourceforge.net Mon Apr 23 23:26:57 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 14:26:57 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 23:23 Message generated for change (Comment added) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) >Assigned to: Guido van Rossum (gvanrossum) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 15:26 Message: Logged In: YES user_id=12364 Originator: YES Neal: Comment added. regrtest -R and -u all report no problems specific to this patch. A clean build of trunk does show problems without it, with test_structmembers, test_tcl, test_telnetlib, test_unicode_file, and test_uuid. At this point I've seen so many weird failures that I think I've started repressing them. ;) When no stack frame is available warn() falls back to sys as the module. Since I'm trying to avoid claiming a warning came from a different file, I made it appear to come from instead. Looking at it now though I wonder if I should go a step further, not having a __warningregistry__ or module at all for such warnings. Guido: I see what you mean about stale co_filename attributes. I'm not sure what the best fix is now. I'm not exactly enthused at having to learn how the import mechanisms work. ;) File Added: python2.6-warningfilename5.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-23 11:31 Message: Logged In: YES user_id=6380 Originator: NO The main reason for using __file__ instead of the filename baked into the code object is that sometimes .pyc files are moved after being compiled; the code object is not updated, but __file__ has the correct path as it reflects how the module was found during import. Also, code objects may have relative pathnames. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 00:15 Message: Logged In: YES user_id=33168 Originator: NO I'm not sure why all the code was in there to begin with it was ancient history. Hopefully Guido remembers. Guido, do you see any particular issue with this approach (ie, removing all the gyrations in warnings and using f_code.co_filename? There is a difference in behaviour. If a module modifies its __file__, the old code would pick up the modified version. This new code will find the real file. I'm not sure if that's a bug or a feature though. :-) Similarly with __name__. Adam, I like adding the warning_guard using a with statement. Although I don't think comment should be completely removed. Could you add a comment about why the guard is necessary (ie, the repeated calls/-R part)? Have you tested with -R and with -u all? ie: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: and ./python -E -tt ./Lib/test/regrtest.py -u all (two separate runs). In the last test case you are replacing (the old spam7), why does "sys" become "no context"? ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-15 19:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 15:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 01:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Tue Apr 24 00:01:42 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 15:01:42 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 23:23 Message generated for change (Comment added) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Guido van Rossum (gvanrossum) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 16:01 Message: Logged In: YES user_id=12364 Originator: YES I "forgot", test_bsddb3 has problems too, but again they're not related to my patch. ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 15:26 Message: Logged In: YES user_id=12364 Originator: YES Neal: Comment added. regrtest -R and -u all report no problems specific to this patch. A clean build of trunk does show problems without it, with test_structmembers, test_tcl, test_telnetlib, test_unicode_file, and test_uuid. At this point I've seen so many weird failures that I think I've started repressing them. ;) When no stack frame is available warn() falls back to sys as the module. Since I'm trying to avoid claiming a warning came from a different file, I made it appear to come from instead. Looking at it now though I wonder if I should go a step further, not having a __warningregistry__ or module at all for such warnings. Guido: I see what you mean about stale co_filename attributes. I'm not sure what the best fix is now. I'm not exactly enthused at having to learn how the import mechanisms work. ;) File Added: python2.6-warningfilename5.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-23 11:31 Message: Logged In: YES user_id=6380 Originator: NO The main reason for using __file__ instead of the filename baked into the code object is that sometimes .pyc files are moved after being compiled; the code object is not updated, but __file__ has the correct path as it reflects how the module was found during import. Also, code objects may have relative pathnames. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 00:15 Message: Logged In: YES user_id=33168 Originator: NO I'm not sure why all the code was in there to begin with it was ancient history. Hopefully Guido remembers. Guido, do you see any particular issue with this approach (ie, removing all the gyrations in warnings and using f_code.co_filename? There is a difference in behaviour. If a module modifies its __file__, the old code would pick up the modified version. This new code will find the real file. I'm not sure if that's a bug or a feature though. :-) Similarly with __name__. Adam, I like adding the warning_guard using a with statement. Although I don't think comment should be completely removed. Could you add a comment about why the guard is necessary (ie, the repeated calls/-R part)? Have you tested with -R and with -u all? ie: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: and ./python -E -tt ./Lib/test/regrtest.py -u all (two separate runs). In the last test case you are replacing (the old spam7), why does "sys" become "no context"? ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-15 19:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 15:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 01:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Tue Apr 24 01:27:45 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 16:27:45 -0700 Subject: [Patches] [ python-Patches-1706039 ] Added clearerr() to clear EOF state Message-ID: Patches item #1706039, was opened at 2007-04-23 17:21 Message generated for change (Comment added) made by josm You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: jos (josm) Assigned to: Nobody/Anonymous (nobody) Summary: Added clearerr() to clear EOF state Initial Comment: This patch is to fix bug 1523853. Now file.read() works as written in the doc, which says "An empty string is returned when EOF is encountered immediately." Before this fix, An empty string was returned even when the last read() encountered EOF. ---------------------------------------------------------------------- >Comment By: jos (josm) Date: 2007-04-23 23:27 Message: Logged In: YES user_id=1776568 Originator: YES I can easily reproduce this problem on OS X 10.4.9 using Python 2.4.3 and Python 2.5. Here's the code ################# import os, sys filename = "spam" f = open(filename, "w+") f.seek(0, 2) line = f.read() # EOF # Writing the same file using another fd f2 = open(filename, "a+") f2.write("Spam") f2.flush() f2.close() statinfo = os.stat(filename) print "file size: %d" % statinfo.st_size print "position : %d" % f.tell() line = f.read() # read() returns emtpy!! readlines?() works ok print "line : [%s]" % line ################# On my machine, it outputs the following ### file size: 4 position : 0 line : [] ### And just now I found that on ubuntu on the same machine (vmware), it works collect. ### file size: 4 position : 0 line : [Spam] ### My patched version python outputs the same result as above on OS X. We need clearerr() there, too because clearerr()'s job is not only only clearing error indicators but EOF indicators also. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-23 20:22 Message: Logged In: YES user_id=835142 Originator: NO I was not able to reproduce the problem. I tried to read through the end of the file, append to the file from an external process and read again. It worked as expected: more data was read. The proposed patch seems to be redundant: once ferror(f->f_fp) is known to be 0, clearerr will do nothing and in the alternative branch clearerr is already there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 From noreply at sourceforge.net Tue Apr 24 02:39:16 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 17:39:16 -0700 Subject: [Patches] [ python-Patches-1703379 ] Refactor test_frozen.py to use unittest. Message-ID: Patches item #1703379, was opened at 2007-04-18 21:49 Message generated for change (Comment added) made by jseutter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jerry Seutter (jseutter) Assigned to: Neal Norwitz (nnorwitz) Summary: Refactor test_frozen.py to use unittest. Initial Comment: This patch refactors test_frozen.py to use unittest. When this patch is applied, Lib/test/output/test_frozen can be removed. I am not sure if the last test case is named appropriately. I named it testImportNonexistentFunction because I believe the code is trying to import a function __phello__.foo, which does not exist. I may be mistaken. ---------------------------------------------------------------------- >Comment By: Jerry Seutter (jseutter) Date: 2007-04-23 18:39 Message: Logged In: YES user_id=1727609 Originator: YES Thanks for the input, guys. See version 2 of the patch which should resolve these issues. Question: Why doesn't regrtest give an error if it doesn't run tests? File Added: refactor_test_frozen.py.patch ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2007-04-19 00:18 Message: Logged In: YES user_id=580910 Originator: NO As Neal wrote sys.platform == 'mac' is of OS9, which isn't supported anymore by Python (or Apple for that matter). The special-case for platform == 'mac' should be removed altoghether and not replaced by a check for OSX because OSX is basically just another unix platform and should therefore support freezing. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 00:08 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! It looks pretty good, but there is one problem I see. There should be a test_main() function for this to work from regrtest. For example, take a look in Lib/test/test_bool.py. Also, I don't think you need to seek(0) before getting the value from sys.stdout (ie, cStringIO instance). I don't think you need to support sys.platform == 'mac'. That was for Mac OS 9 which is no longer supported (see PEP 11). Mac OS 10 has sys.platform == 'darwin'. If you're interested, it would be good to produce a patch to remove all (10+) occurrences of comparing sys.platform with mac. I'm assigning this to Ronald just so he can verify the Mac OS 9 issue. Ronald, please comment and then unassign or assign to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 From noreply at sourceforge.net Tue Apr 24 05:37:26 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 20:37:26 -0700 Subject: [Patches] [ python-Patches-1703379 ] Refactor test_frozen.py to use unittest. Message-ID: Patches item #1703379, was opened at 2007-04-18 20:49 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jerry Seutter (jseutter) Assigned to: Neal Norwitz (nnorwitz) Summary: Refactor test_frozen.py to use unittest. Initial Comment: This patch refactors test_frozen.py to use unittest. When this patch is applied, Lib/test/output/test_frozen can be removed. I am not sure if the last test case is named appropriately. I named it testImportNonexistentFunction because I believe the code is trying to import a function __phello__.foo, which does not exist. I may be mistaken. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 20:37 Message: Logged In: YES user_id=33168 Originator: NO Thanks for updating the patch, I'll try to take a look at it soon (within a week or so). If anyone else wants to take a look at this, feel free. > Why doesn't regrtest give an error if it doesn't run tests? regrtest can't know the difference between an old style test which executes on import and a new style test which has a test_main. This is why I want to get rid of all the old style tests (the ones that execute by side-effect of import). Then we can have regrtest complain if there is no test_main method just as you suggest. ---------------------------------------------------------------------- Comment By: Jerry Seutter (jseutter) Date: 2007-04-23 17:39 Message: Logged In: YES user_id=1727609 Originator: YES Thanks for the input, guys. See version 2 of the patch which should resolve these issues. Question: Why doesn't regrtest give an error if it doesn't run tests? File Added: refactor_test_frozen.py.patch ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2007-04-18 23:18 Message: Logged In: YES user_id=580910 Originator: NO As Neal wrote sys.platform == 'mac' is of OS9, which isn't supported anymore by Python (or Apple for that matter). The special-case for platform == 'mac' should be removed altoghether and not replaced by a check for OSX because OSX is basically just another unix platform and should therefore support freezing. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-18 23:08 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! It looks pretty good, but there is one problem I see. There should be a test_main() function for this to work from regrtest. For example, take a look in Lib/test/test_bool.py. Also, I don't think you need to seek(0) before getting the value from sys.stdout (ie, cStringIO instance). I don't think you need to support sys.platform == 'mac'. That was for Mac OS 9 which is no longer supported (see PEP 11). Mac OS 10 has sys.platform == 'darwin'. If you're interested, it would be good to produce a patch to remove all (10+) occurrences of comparing sys.platform with mac. I'm assigning this to Ronald just so he can verify the Mac OS 9 issue. Ronald, please comment and then unassign or assign to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 From noreply at sourceforge.net Tue Apr 24 06:28:09 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 21:28:09 -0700 Subject: [Patches] [ python-Patches-1706039 ] Added clearerr() to clear EOF state Message-ID: Patches item #1706039, was opened at 2007-04-23 13:21 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: jos (josm) Assigned to: Nobody/Anonymous (nobody) Summary: Added clearerr() to clear EOF state Initial Comment: This patch is to fix bug 1523853. Now file.read() works as written in the doc, which says "An empty string is returned when EOF is encountered immediately." Before this fix, An empty string was returned even when the last read() encountered EOF. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-24 00:28 Message: Logged In: YES user_id=835142 Originator: NO Hmm. It looks like glibc's fread will attempt reading even if EOF flag is already set. I've reproduced the problem on OS X 10.4.9 and Python 2.6a0 (trunk:54926M). This patch fixes it. ---------------------------------------------------------------------- Comment By: jos (josm) Date: 2007-04-23 19:27 Message: Logged In: YES user_id=1776568 Originator: YES I can easily reproduce this problem on OS X 10.4.9 using Python 2.4.3 and Python 2.5. Here's the code ################# import os, sys filename = "spam" f = open(filename, "w+") f.seek(0, 2) line = f.read() # EOF # Writing the same file using another fd f2 = open(filename, "a+") f2.write("Spam") f2.flush() f2.close() statinfo = os.stat(filename) print "file size: %d" % statinfo.st_size print "position : %d" % f.tell() line = f.read() # read() returns emtpy!! readlines?() works ok print "line : [%s]" % line ################# On my machine, it outputs the following ### file size: 4 position : 0 line : [] ### And just now I found that on ubuntu on the same machine (vmware), it works collect. ### file size: 4 position : 0 line : [Spam] ### My patched version python outputs the same result as above on OS X. We need clearerr() there, too because clearerr()'s job is not only only clearing error indicators but EOF indicators also. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-23 16:22 Message: Logged In: YES user_id=835142 Originator: NO I was not able to reproduce the problem. I tried to read through the end of the file, append to the file from an external process and read again. It worked as expected: more data was read. The proposed patch seems to be redundant: once ferror(f->f_fp) is known to be 0, clearerr will do nothing and in the alternative branch clearerr is already there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 From noreply at sourceforge.net Tue Apr 24 07:04:52 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 22:04:52 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 22:23 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) >Assigned to: Neal Norwitz (nnorwitz) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 22:04 Message: Logged In: YES user_id=33168 Originator: NO Unfortunately, test_bsddb3 has lots of problems. :-( They vary by platform and version of bsddb used. Given the current code works, I'm not sure it's worth spending time cleaning this up. Especially since I plan to move this code to C. It might be worthwhile to separate out the warning guard. I can't remember if the __warningregistry__ was mucked with in multiple places. If it's only used in the one place, I'm not sure that's worth it either. Learning import is a test of manhood. :-) ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 15:01 Message: Logged In: YES user_id=12364 Originator: YES I "forgot", test_bsddb3 has problems too, but again they're not related to my patch. ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 14:26 Message: Logged In: YES user_id=12364 Originator: YES Neal: Comment added. regrtest -R and -u all report no problems specific to this patch. A clean build of trunk does show problems without it, with test_structmembers, test_tcl, test_telnetlib, test_unicode_file, and test_uuid. At this point I've seen so many weird failures that I think I've started repressing them. ;) When no stack frame is available warn() falls back to sys as the module. Since I'm trying to avoid claiming a warning came from a different file, I made it appear to come from instead. Looking at it now though I wonder if I should go a step further, not having a __warningregistry__ or module at all for such warnings. Guido: I see what you mean about stale co_filename attributes. I'm not sure what the best fix is now. I'm not exactly enthused at having to learn how the import mechanisms work. ;) File Added: python2.6-warningfilename5.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-23 10:31 Message: Logged In: YES user_id=6380 Originator: NO The main reason for using __file__ instead of the filename baked into the code object is that sometimes .pyc files are moved after being compiled; the code object is not updated, but __file__ has the correct path as it reflects how the module was found during import. Also, code objects may have relative pathnames. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-22 23:15 Message: Logged In: YES user_id=33168 Originator: NO I'm not sure why all the code was in there to begin with it was ancient history. Hopefully Guido remembers. Guido, do you see any particular issue with this approach (ie, removing all the gyrations in warnings and using f_code.co_filename? There is a difference in behaviour. If a module modifies its __file__, the old code would pick up the modified version. This new code will find the real file. I'm not sure if that's a bug or a feature though. :-) Similarly with __name__. Adam, I like adding the warning_guard using a with statement. Although I don't think comment should be completely removed. Could you add a comment about why the guard is necessary (ie, the repeated calls/-R part)? Have you tested with -R and with -u all? ie: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: and ./python -E -tt ./Lib/test/regrtest.py -u all (two separate runs). In the last test case you are replacing (the old spam7), why does "sys" become "no context"? ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-15 18:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 14:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 00:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Tue Apr 24 07:30:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 22:30:12 -0700 Subject: [Patches] [ python-Patches-1706323 ] Updated ASTVisitor Classes Message-ID: Patches item #1706323, was opened at 2007-04-24 07:30 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Yaakov Nemoy (loupgaroublond) Assigned to: Nobody/Anonymous (nobody) Summary: Updated ASTVisitor Classes Initial Comment: Ok, So this is really more of a complete rewrite than a bunch of patches, but it's just a picture of what IMHO is a saner way to do this module. I suppose I could refactor alot of it to look more like the old module, but I'm still not sure what would be better, in terms of having seperate visitors to walkers. Either way, the terminology in the original wasn't much clearer. At bare minimum though, i got postorder implemented, so this code has got to be worth something valuable... I hope.... Feedback please. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 From noreply at sourceforge.net Tue Apr 24 07:57:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 22:57:33 -0700 Subject: [Patches] [ python-Patches-1671298 ] Refactor test_class to use unittest lib Message-ID: Patches item #1671298, was opened at 2007-02-28 15:38 Message generated for change (Comment added) made by jyzude You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1671298&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Mike Verdone (jyzude) Assigned to: Collin Winter (collinwinter) Summary: Refactor test_class to use unittest lib Initial Comment: Refactored Lib/test/test_class.py to use unittest library instead of icky output comparison tests. Also have to delete output/test_class after adding this patch. ---------------------------------------------------------------------- >Comment By: Mike Verdone (jyzude) Date: 2007-04-23 23:57 Message: Logged In: YES user_id=584997 Originator: YES Hi all, Good catch Neal, I needed a test_main method. I also finally got around to tightening up the tests so that at all times the entire call stack is tested. It's a bit messier than before and somewhat brittle, but it's thorough and checks every ugly implicit call to __coerce__ just like the old tests did. Only problem is that now the tests fail. The test of the statement "1 == testme" on line 419 generates a call stack that seems very strange and I can't figure out what it means. It might be a bug in the python interpreter... or a feature... or just a mistake in the test. At this point I can't figure it out but I'll post my patch so far. If anyone can figure out what's going on please let me know! File Added: test_class.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 00:17 Message: Logged In: YES user_id=33168 Originator: NO There should be a test_main method otherwise this won't work when called from regrtest. Have you tried to run this with regrtest -R :: ? I think it will work, but just wanted to be sure. One thing that might be nice to add to this test is to verify the change in the length of the callLst since assertLastCallWas() was called. Typically there should be only one method call from what I saw in the test. However, if we screw something up and there are two method calls, this test could catch that. It would be an enhancement (ie new feature) over the existing test. ---------------------------------------------------------------------- Comment By: Jerry Seutter (jseutter) Date: 2007-04-18 11:45 Message: Logged In: YES user_id=1727609 Originator: NO This patch looks good to me. ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-19 16:23 Message: Logged In: YES user_id=584997 Originator: YES Hi Collin, Sorry for the delay. Your tweaks look very good. I don't see any problems. ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-11 11:42 Message: Logged In: YES user_id=1344176 Originator: NO Mike, I've tweaked your refactoring some: strengthened the assertions in testMixIntsAndLongs and testDel; code cleanup in testBadTypeReturned; removed Jython-related code; reduced line lengths to < 80; changed a few print statements to self.fail() calls. Look over the version I've uploaded and tell me what you think. File Added: test_class.py.diff ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-07 14:07 Message: Logged In: YES user_id=584997 Originator: YES Removed unnecessary global statement in trackCall. Anything else? :-) File Added: test_class.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-06 18:03 Message: Logged In: YES user_id=849994 Originator: NO Note that you don't need the global statement for callLst as you aren't rebinding it in the function. ---------------------------------------------------------------------- Comment By: Mike Verdone (jyzude) Date: 2007-03-05 23:38 Message: Logged In: YES user_id=584997 Originator: YES Hi collin, * I improved testMixIntsAndLongs. It now asserts things * assert is banished, replaced by the correct calls * the reason why callLst is global is because I have to track calls to __getitem__ in some cases. Because of this, if I put callLst on the object I end up with horrible recursive loops, or at the very least the last call on the stack will always be __getitem__ when I get the list to inspect. * assertLastCall only checks the last thing on the list because generally the thing called before that is always __getitem__ or associated magic. I don't want my tests to be bound to the internals of __getitem__. All I care about is that ultimately the right function was called. That said, I modified assertLastCallWas to erase the callLst to prevent any possible bleed-over from the previous test. Let me know if you have further suggestions. File Added: test_class.patch ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2007-03-05 17:11 Message: Logged In: YES user_id=1344176 Originator: NO Thanks for your effort! This generally looks good. A few minor things: - testMixIntsAndLongs doesn't have any assertions in it. - "assert" statements should probably be changed to use the failUnlessEqual/assertEqual methods. - I'm wary of your assertLastCallWas system; I'd feel more comfortable if you were making assertions about the entire call chain, not just its last item. Also, something like callLst feels strange as a global. Feel free to contact me to discuss this off-list. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1671298&group_id=5470 From noreply at sourceforge.net Tue Apr 24 08:12:34 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 23:12:34 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-01 23:23 Message generated for change (Comment added) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Closed Resolution: Rejected Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Neal Norwitz (nnorwitz) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-24 00:12 Message: Logged In: YES user_id=12364 Originator: YES test_struct is the only thing that touches __warningregistry__ besides test_warning and (in my other patch) test_py3kwarnings. Looks like it would benefit from it, and could use some decorators too.. def check_float_coerce(format, number): ... check_float_coerce = with_warning_restore(deprecated_err) *cough* Paraphrases of religious quotes relating to the creation of decorators aside, I think that'd be better written as a contextmanager. By the time I'm done I'm going to end up with two dozen patch sets covering 80% of the python codebase x_x. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 23:04 Message: Logged In: YES user_id=33168 Originator: NO Unfortunately, test_bsddb3 has lots of problems. :-( They vary by platform and version of bsddb used. Given the current code works, I'm not sure it's worth spending time cleaning this up. Especially since I plan to move this code to C. It might be worthwhile to separate out the warning guard. I can't remember if the __warningregistry__ was mucked with in multiple places. If it's only used in the one place, I'm not sure that's worth it either. Learning import is a test of manhood. :-) ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 16:01 Message: Logged In: YES user_id=12364 Originator: YES I "forgot", test_bsddb3 has problems too, but again they're not related to my patch. ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 15:26 Message: Logged In: YES user_id=12364 Originator: YES Neal: Comment added. regrtest -R and -u all report no problems specific to this patch. A clean build of trunk does show problems without it, with test_structmembers, test_tcl, test_telnetlib, test_unicode_file, and test_uuid. At this point I've seen so many weird failures that I think I've started repressing them. ;) When no stack frame is available warn() falls back to sys as the module. Since I'm trying to avoid claiming a warning came from a different file, I made it appear to come from instead. Looking at it now though I wonder if I should go a step further, not having a __warningregistry__ or module at all for such warnings. Guido: I see what you mean about stale co_filename attributes. I'm not sure what the best fix is now. I'm not exactly enthused at having to learn how the import mechanisms work. ;) File Added: python2.6-warningfilename5.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-23 11:31 Message: Logged In: YES user_id=6380 Originator: NO The main reason for using __file__ instead of the filename baked into the code object is that sometimes .pyc files are moved after being compiled; the code object is not updated, but __file__ has the correct path as it reflects how the module was found during import. Also, code objects may have relative pathnames. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 00:15 Message: Logged In: YES user_id=33168 Originator: NO I'm not sure why all the code was in there to begin with it was ancient history. Hopefully Guido remembers. Guido, do you see any particular issue with this approach (ie, removing all the gyrations in warnings and using f_code.co_filename? There is a difference in behaviour. If a module modifies its __file__, the old code would pick up the modified version. This new code will find the real file. I'm not sure if that's a bug or a feature though. :-) Similarly with __name__. Adam, I like adding the warning_guard using a with statement. Although I don't think comment should be completely removed. Could you add a comment about why the guard is necessary (ie, the repeated calls/-R part)? Have you tested with -R and with -u all? ie: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: and ./python -E -tt ./Lib/test/regrtest.py -u all (two separate runs). In the last test case you are replacing (the old spam7), why does "sys" become "no context"? ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-15 19:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 15:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 01:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Tue Apr 24 08:58:14 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 23 Apr 2007 23:58:14 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 00:39 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 23:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 06:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Tue Apr 24 12:06:38 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 03:06:38 -0700 Subject: [Patches] [ python-Patches-1692664 ] warnings.py gets filename wrong for eval/exec Message-ID: Patches item #1692664, was opened at 2007-04-02 07:23 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Closed Resolution: Rejected Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Neal Norwitz (nnorwitz) Summary: warnings.py gets filename wrong for eval/exec Initial Comment: warnings.warn() gets the filename using the globals' __file__ attribute. When using eval or exec this is often not the context in which the current line was compiled from. The line number is correct, but will be applied to whatever file the globals are from, leading to an unrelated line being printed below the warning. The attached patch makes it use caller.f_code.co_filename instead. This also seems to remove the need to normalize .pyc/.pyo files, as well as not needing to use sys.argv[0] for __main__ modules. It also cleans up warnings.warn() and adds three unit tests. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-24 12:06 Message: Logged In: YES user_id=1326842 Originator: NO Bug #1180193 has a patch with a fix for stale co_filename attributes: http://www.python.org/sf/1180193 ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-24 08:12 Message: Logged In: YES user_id=12364 Originator: YES test_struct is the only thing that touches __warningregistry__ besides test_warning and (in my other patch) test_py3kwarnings. Looks like it would benefit from it, and could use some decorators too.. def check_float_coerce(format, number): ... check_float_coerce = with_warning_restore(deprecated_err) *cough* Paraphrases of religious quotes relating to the creation of decorators aside, I think that'd be better written as a contextmanager. By the time I'm done I'm going to end up with two dozen patch sets covering 80% of the python codebase x_x. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 07:04 Message: Logged In: YES user_id=33168 Originator: NO Unfortunately, test_bsddb3 has lots of problems. :-( They vary by platform and version of bsddb used. Given the current code works, I'm not sure it's worth spending time cleaning this up. Especially since I plan to move this code to C. It might be worthwhile to separate out the warning guard. I can't remember if the __warningregistry__ was mucked with in multiple places. If it's only used in the one place, I'm not sure that's worth it either. Learning import is a test of manhood. :-) ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-24 00:01 Message: Logged In: YES user_id=12364 Originator: YES I "forgot", test_bsddb3 has problems too, but again they're not related to my patch. ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-23 23:26 Message: Logged In: YES user_id=12364 Originator: YES Neal: Comment added. regrtest -R and -u all report no problems specific to this patch. A clean build of trunk does show problems without it, with test_structmembers, test_tcl, test_telnetlib, test_unicode_file, and test_uuid. At this point I've seen so many weird failures that I think I've started repressing them. ;) When no stack frame is available warn() falls back to sys as the module. Since I'm trying to avoid claiming a warning came from a different file, I made it appear to come from instead. Looking at it now though I wonder if I should go a step further, not having a __warningregistry__ or module at all for such warnings. Guido: I see what you mean about stale co_filename attributes. I'm not sure what the best fix is now. I'm not exactly enthused at having to learn how the import mechanisms work. ;) File Added: python2.6-warningfilename5.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-23 19:31 Message: Logged In: YES user_id=6380 Originator: NO The main reason for using __file__ instead of the filename baked into the code object is that sometimes .pyc files are moved after being compiled; the code object is not updated, but __file__ has the correct path as it reflects how the module was found during import. Also, code objects may have relative pathnames. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 08:15 Message: Logged In: YES user_id=33168 Originator: NO I'm not sure why all the code was in there to begin with it was ancient history. Hopefully Guido remembers. Guido, do you see any particular issue with this approach (ie, removing all the gyrations in warnings and using f_code.co_filename? There is a difference in behaviour. If a module modifies its __file__, the old code would pick up the modified version. This new code will find the real file. I'm not sure if that's a bug or a feature though. :-) Similarly with __name__. Adam, I like adding the warning_guard using a with statement. Although I don't think comment should be completely removed. Could you add a comment about why the guard is necessary (ie, the repeated calls/-R part)? Have you tested with -R and with -u all? ie: ./python -E -tt ./Lib/test/regrtest.py -R 4:3: and ./python -E -tt ./Lib/test/regrtest.py -u all (two separate runs). In the last test case you are replacing (the old spam7), why does "sys" become "no context"? ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-16 03:34 Message: Logged In: YES user_id=12364 Originator: YES I've rewritten the unit tests I added to follow the style of walter.doerwald's changes to test_warnings.py (commited to SVN during the same period in which I posted my patch). The guard_warnings_registry() context manager is now in test_support. File Added: python2.6-warningfilename4.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-04 23:28 Message: Logged In: YES user_id=12364 Originator: YES sys._getframe(sys.maxint) produces an OverflowError on 64bit GCC. Patch changed to use sys._getframe(10**9) instead. File Added: python2.6-warningfilename3.diff ---------------------------------------------------------------------- Comment By: Adam Olsen (rhamphoryncus) Date: 2007-04-03 09:00 Message: Logged In: YES user_id=12364 Originator: YES The test_stackoverflow_message test I added causes warnings.py to use sys.__warningregistry__ rather than test_warnings.__warningregistry__. This circumvents this self-declared hack of deleting test_warnings.__warningregistry__ to allow regrtest -R to run. This version of the patch uses a more general solution to allowing regrtest -R to run. Probably doesn't qualify as a hack anymore... File Added: python2.6-warningfilename2.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1692664&group_id=5470 From noreply at sourceforge.net Tue Apr 24 12:32:52 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 03:32:52 -0700 Subject: [Patches] [ python-Patches-1686487 ] Allow any mapping after ** in calls Message-ID: Patches item #1686487, was opened at 2007-03-23 04:27 Message generated for change (Comment added) made by zseil You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: Allow any mapping after ** in calls Initial Comment: Python allows arbitrary sequences after * in calls, but an expression following ** must be a (subclass of) dict. The attached patch makes Python accept arbitrary mappings after **. ---------------------------------------------------------------------- >Comment By: Ziga Seilnacht (zseil) Date: 2007-04-24 12:32 Message: Logged In: YES user_id=1326842 Originator: NO The patch looks good to me. The only thing that could be improved is the error message. It would be nice if it contained the information about kwdict's type. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 21:53 Message: Logged In: YES user_id=835142 Originator: YES File Added: star-star-mapping.patch ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 21:38 Message: Logged In: YES user_id=835142 Originator: YES The proposed patch handles ** arguments similar to the way * arguments are treated. If the expression following ** evaluates to an object that is not a dictionary, the new code attempts to update a new dictionary with that object. This procedure is closely similar to what PySequence_Tuple does with * arguments. (Python API does not provide PyMapping_Dict.) If new dictionary is succesfully created and updated, it replaces the original ** argument. If that attempt fails because kwdict is not a mapping (does not have a 'keys' attribute, a type error is raised explaining that a mapping is expected. All other errors, e.g. MemoryError are percolated up. (I am am removing the previous versions of the patch.) File Added: star-star-mapping.patch ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 20:34 Message: Logged In: YES user_id=835142 Originator: YES Georg, Thanks for reposting my proposal. Since BDFL is +1 on the idea, I feel motivated to polish the patch. In your early comment you noted that kwdict == NULL check may be superficial. I agree. It does not look like it is possible that ext_do_call is called with CALL_FLAG_KW set and a null pointer at the top of the stack. It looks like this may only happen if the stack is corrupted by a misbehaving c extension. In this case, however, throwing a type error is too gentle. PyErr_BadInternalCall() seems to be more appropriate if anything at all. I propose to eliminate the null check for kwdict alltogether. This would be consistent with the way stararg is handled later in this function. I am going to run the testsuit without the null check and will post a revised patch shortly. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 18:02 Message: Logged In: YES user_id=835142 Originator: YES I have asked on python-dev almost a month ago, but did not get any response: http://mail.python.org/pipermail/python-dev/2007-March/072321.html I have recently posted another message that got no response as well: http://mail.python.org/pipermail/python-dev/2007-April/072653.html Is it possible that my messages are not being delivered to the list? ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-03-23 20:44 Message: Logged In: YES user_id=835142 Originator: YES I've fixed the flaws in star-star-mapping-1.patch . I will ask on python-dev about the feature. If the proposal is accepted, I will probably refactor the code to avoid using the second goto label. File Added: star-star-mapping-1.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 19:45 Message: Logged In: YES user_id=849994 Originator: NO Two flaws: - what if kwdict is NULL? Shouldn't happen, but the old version raises an error in this case. - if PyDict_Update fails, tmpdict is leaked. Please ask on python-dev whether to include this feature. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 From noreply at sourceforge.net Tue Apr 24 13:53:03 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 04:53:03 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 03:39 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-24 07:53 Message: Logged In: YES user_id=18139 Originator: YES Neal, thanks for looking at this. Size: Yeah. :-\ I don't know what to do about this. There were a lot of bugs, so there are a lot of diffs. :( Whitespace: I checked, and the extra whitespace is definitely in trunk's Lib/test/test_minidom.py; a recent change added it. This patch removes it. (I could revert the whitespace anyway, just to reduce the size of the patch. Let me know if I should do this.) Doc: Oops! I will fix this today and upload a new patch. pass: Thanks, I'll change as suggested. (Just to explain myself-- I used "if name == 'nodeValue': pass; else: ..." because 'nodeValue' is the special case, and the special behavior is "do nothing". But I'm not attached to that wording. I'll change it.) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 02:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 09:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Tue Apr 24 15:19:39 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 06:19:39 -0700 Subject: [Patches] [ python-Patches-1686487 ] Allow any mapping after ** in calls Message-ID: Patches item #1686487, was opened at 2007-03-22 23:27 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belopolsky (belopolsky) Assigned to: Nobody/Anonymous (nobody) Summary: Allow any mapping after ** in calls Initial Comment: Python allows arbitrary sequences after * in calls, but an expression following ** must be a (subclass of) dict. The attached patch makes Python accept arbitrary mappings after **. ---------------------------------------------------------------------- >Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-24 09:19 Message: Logged In: YES user_id=835142 Originator: YES zseil: It would be nice if it contained the information about kwdict's type. I agree, the new patch adds type information to both * and ** error messages. File Added: star-star-mapping-msg.patch ---------------------------------------------------------------------- Comment By: Ziga Seilnacht (zseil) Date: 2007-04-24 06:32 Message: Logged In: YES user_id=1326842 Originator: NO The patch looks good to me. The only thing that could be improved is the error message. It would be nice if it contained the information about kwdict's type. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 15:53 Message: Logged In: YES user_id=835142 Originator: YES File Added: star-star-mapping.patch ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 15:38 Message: Logged In: YES user_id=835142 Originator: YES The proposed patch handles ** arguments similar to the way * arguments are treated. If the expression following ** evaluates to an object that is not a dictionary, the new code attempts to update a new dictionary with that object. This procedure is closely similar to what PySequence_Tuple does with * arguments. (Python API does not provide PyMapping_Dict.) If new dictionary is succesfully created and updated, it replaces the original ** argument. If that attempt fails because kwdict is not a mapping (does not have a 'keys' attribute, a type error is raised explaining that a mapping is expected. All other errors, e.g. MemoryError are percolated up. (I am am removing the previous versions of the patch.) File Added: star-star-mapping.patch ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 14:34 Message: Logged In: YES user_id=835142 Originator: YES Georg, Thanks for reposting my proposal. Since BDFL is +1 on the idea, I feel motivated to polish the patch. In your early comment you noted that kwdict == NULL check may be superficial. I agree. It does not look like it is possible that ext_do_call is called with CALL_FLAG_KW set and a null pointer at the top of the stack. It looks like this may only happen if the stack is corrupted by a misbehaving c extension. In this case, however, throwing a type error is too gentle. PyErr_BadInternalCall() seems to be more appropriate if anything at all. I propose to eliminate the null check for kwdict alltogether. This would be consistent with the way stararg is handled later in this function. I am going to run the testsuit without the null check and will post a revised patch shortly. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-18 12:02 Message: Logged In: YES user_id=835142 Originator: YES I have asked on python-dev almost a month ago, but did not get any response: http://mail.python.org/pipermail/python-dev/2007-March/072321.html I have recently posted another message that got no response as well: http://mail.python.org/pipermail/python-dev/2007-April/072653.html Is it possible that my messages are not being delivered to the list? ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-03-23 15:44 Message: Logged In: YES user_id=835142 Originator: YES I've fixed the flaws in star-star-mapping-1.patch . I will ask on python-dev about the feature. If the proposal is accepted, I will probably refactor the code to avoid using the second goto label. File Added: star-star-mapping-1.patch ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-03-23 14:45 Message: Logged In: YES user_id=849994 Originator: NO Two flaws: - what if kwdict is NULL? Shouldn't happen, but the old version raises an error in this case. - if PyDict_Update fails, tmpdict is leaked. Please ask on python-dev whether to include this feature. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1686487&group_id=5470 From noreply at sourceforge.net Tue Apr 24 15:53:30 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 06:53:30 -0700 Subject: [Patches] [ python-Patches-1706323 ] Updated ASTVisitor Classes Message-ID: Patches item #1706323, was opened at 2007-04-24 01:30 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Yaakov Nemoy (loupgaroublond) Assigned to: Nobody/Anonymous (nobody) Summary: Updated ASTVisitor Classes Initial Comment: Ok, So this is really more of a complete rewrite than a bunch of patches, but it's just a picture of what IMHO is a saner way to do this module. I suppose I could refactor alot of it to look more like the old module, but I'm still not sure what would be better, in terms of having seperate visitors to walkers. Either way, the terminology in the original wasn't much clearer. At bare minimum though, i got postorder implemented, so this code has got to be worth something valuable... I hope.... Feedback please. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 09:53 Message: Logged In: YES user_id=764593 Originator: NO I haven't looked deeply yet, but I know some feedback quickly is often better than a long wait. I do have a few style suggestions. (1) Tests for None should generally use "is" rather than "==". (2) It helps to understand the code if it is more parallel. Seeing the meth is/is not None cases reversed on the two orders confused me about how similar they were. (3) Inheriting from object (or setting __metaclass__ = type) is probably a good habit, since old-style classes will disappear in Py3K. Normally, it doesn't matter, but debugging the difference is a pain. To make the first two concrete, here is my rewording of ASTVisitor.dispatch: From: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if self._order == self.PREORDER: if meth != None and meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) elif meth == None: self._iterate_over_others(node, *args) elif self._order == self.POSTORDER: if meth == None: self._iterate_over_others(node, *args) if meth != None: self._iterate_over_defaults(node, *args) meth(node, *args) To: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if meth is None: self._iterate_over_others(node, *args) else: if self._order == self.POSTORDER: self._iterate_over_defaults(node, *args) meth(node, *args) elif meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) else: pass # pre-order, and the method said to stop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 From noreply at sourceforge.net Tue Apr 24 19:15:50 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 10:15:50 -0700 Subject: [Patches] [ python-Patches-1706039 ] Added clearerr() to clear EOF state Message-ID: Patches item #1706039, was opened at 2007-04-23 17:21 Message generated for change (Comment added) made by josm You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: jos (josm) Assigned to: Nobody/Anonymous (nobody) Summary: Added clearerr() to clear EOF state Initial Comment: This patch is to fix bug 1523853. Now file.read() works as written in the doc, which says "An empty string is returned when EOF is encountered immediately." Before this fix, An empty string was returned even when the last read() encountered EOF. ---------------------------------------------------------------------- >Comment By: jos (josm) Date: 2007-04-24 17:15 Message: Logged In: YES user_id=1776568 Originator: YES I confirmed that fread can read data from fp in EOF state on Linux. On OS X, freading from fp in EOF state doesn't work, just return 0. I tested this behavior using the following code. ---------------------------- #include #include main(void) { FILE *fp1, *fp2; int n; char buf[128]; fp1 = fopen("spam.txt", "w+"); fputs("foo", fp1); fseek(fp1, 0L, SEEK_END); if (fread(buf, 1, sizeof(buf), fp1) == 0) fprintf(stderr, "first fread failed\n"); fp2 = fopen("spam.txt", "a+"); fputs("bar", fp2); fclose(fp2); if (feof(fp1)) fprintf(stderr, "after appending some text, fp1 is still eof\n"); if (fread(buf, 1, sizeof(buf), fp1) == 0) fprintf(stderr, "second fread failed\n"); printf("buf: %s\n", buf); fclose(fp1); return 0; } ---------------------------- ============= On Linux ============= first fread failed after appending some text, fp1 is still eof buf: bar ============= On OS X ============= first fread failed after appending some text, fp1 is still eof second fread failed buf: Anyway, I think it's safe and preferable to clear EOF indicator in this case. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-24 04:28 Message: Logged In: YES user_id=835142 Originator: NO Hmm. It looks like glibc's fread will attempt reading even if EOF flag is already set. I've reproduced the problem on OS X 10.4.9 and Python 2.6a0 (trunk:54926M). This patch fixes it. ---------------------------------------------------------------------- Comment By: jos (josm) Date: 2007-04-23 23:27 Message: Logged In: YES user_id=1776568 Originator: YES I can easily reproduce this problem on OS X 10.4.9 using Python 2.4.3 and Python 2.5. Here's the code ################# import os, sys filename = "spam" f = open(filename, "w+") f.seek(0, 2) line = f.read() # EOF # Writing the same file using another fd f2 = open(filename, "a+") f2.write("Spam") f2.flush() f2.close() statinfo = os.stat(filename) print "file size: %d" % statinfo.st_size print "position : %d" % f.tell() line = f.read() # read() returns emtpy!! readlines?() works ok print "line : [%s]" % line ################# On my machine, it outputs the following ### file size: 4 position : 0 line : [] ### And just now I found that on ubuntu on the same machine (vmware), it works collect. ### file size: 4 position : 0 line : [Spam] ### My patched version python outputs the same result as above on OS X. We need clearerr() there, too because clearerr()'s job is not only only clearing error indicators but EOF indicators also. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-23 20:22 Message: Logged In: YES user_id=835142 Originator: NO I was not able to reproduce the problem. I tried to read through the end of the file, append to the file from an external process and read again. It worked as expected: more data was read. The proposed patch seems to be redundant: once ferror(f->f_fp) is known to be 0, clearerr will do nothing and in the alternative branch clearerr is already there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 From noreply at sourceforge.net Tue Apr 24 23:03:33 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 14:03:33 -0700 Subject: [Patches] [ python-Patches-1706323 ] Updated ASTVisitor Classes Message-ID: Patches item #1706323, was opened at 2007-04-24 07:30 Message generated for change (Comment added) made by loupgaroublond You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Yaakov Nemoy (loupgaroublond) Assigned to: Nobody/Anonymous (nobody) Summary: Updated ASTVisitor Classes Initial Comment: Ok, So this is really more of a complete rewrite than a bunch of patches, but it's just a picture of what IMHO is a saner way to do this module. I suppose I could refactor alot of it to look more like the old module, but I'm still not sure what would be better, in terms of having seperate visitors to walkers. Either way, the terminology in the original wasn't much clearer. At bare minimum though, i got postorder implemented, so this code has got to be worth something valuable... I hope.... Feedback please. ---------------------------------------------------------------------- >Comment By: Yaakov Nemoy (loupgaroublond) Date: 2007-04-24 23:03 Message: Logged In: YES user_id=1487908 Originator: YES That looks like a good idea. I just hammered the code out last night, and I'll try to include your changes tonight if I get a chance. I'll take a look at #3, but I'm not quite familiar with the Py3k changes as I should be. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 15:53 Message: Logged In: YES user_id=764593 Originator: NO I haven't looked deeply yet, but I know some feedback quickly is often better than a long wait. I do have a few style suggestions. (1) Tests for None should generally use "is" rather than "==". (2) It helps to understand the code if it is more parallel. Seeing the meth is/is not None cases reversed on the two orders confused me about how similar they were. (3) Inheriting from object (or setting __metaclass__ = type) is probably a good habit, since old-style classes will disappear in Py3K. Normally, it doesn't matter, but debugging the difference is a pain. To make the first two concrete, here is my rewording of ASTVisitor.dispatch: From: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if self._order == self.PREORDER: if meth != None and meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) elif meth == None: self._iterate_over_others(node, *args) elif self._order == self.POSTORDER: if meth == None: self._iterate_over_others(node, *args) if meth != None: self._iterate_over_defaults(node, *args) meth(node, *args) To: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if meth is None: self._iterate_over_others(node, *args) else: if self._order == self.POSTORDER: self._iterate_over_defaults(node, *args) meth(node, *args) elif meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) else: pass # pre-order, and the method said to stop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 From noreply at sourceforge.net Tue Apr 24 23:16:38 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 14:16:38 -0700 Subject: [Patches] [ python-Patches-1706323 ] Updated ASTVisitor Classes Message-ID: Patches item #1706323, was opened at 2007-04-24 01:30 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Yaakov Nemoy (loupgaroublond) Assigned to: Nobody/Anonymous (nobody) Summary: Updated ASTVisitor Classes Initial Comment: Ok, So this is really more of a complete rewrite than a bunch of patches, but it's just a picture of what IMHO is a saner way to do this module. I suppose I could refactor alot of it to look more like the old module, but I'm still not sure what would be better, in terms of having seperate visitors to walkers. Either way, the terminology in the original wasn't much clearer. At bare minimum though, i got postorder implemented, so this code has got to be worth something valuable... I hope.... Feedback please. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 17:16 Message: Logged In: YES user_id=764593 Originator: NO > I'll take a look at #3, but I'm not quite familiar with the Py3k changes Don't worry too much -- for the most part, Py3k is just a matter of cleanup. For reasonable code, there usually isn't much difference between class Old: ... and class New(object): ... But this means that when there is a difference, it is a pain to debug. So just stick with the version that will continue to be available (inheriting from object), and backwards-compatibility will take care of itself. ---------------------------------------------------------------------- Comment By: Yaakov Nemoy (loupgaroublond) Date: 2007-04-24 17:03 Message: Logged In: YES user_id=1487908 Originator: YES That looks like a good idea. I just hammered the code out last night, and I'll try to include your changes tonight if I get a chance. I'll take a look at #3, but I'm not quite familiar with the Py3k changes as I should be. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 09:53 Message: Logged In: YES user_id=764593 Originator: NO I haven't looked deeply yet, but I know some feedback quickly is often better than a long wait. I do have a few style suggestions. (1) Tests for None should generally use "is" rather than "==". (2) It helps to understand the code if it is more parallel. Seeing the meth is/is not None cases reversed on the two orders confused me about how similar they were. (3) Inheriting from object (or setting __metaclass__ = type) is probably a good habit, since old-style classes will disappear in Py3K. Normally, it doesn't matter, but debugging the difference is a pain. To make the first two concrete, here is my rewording of ASTVisitor.dispatch: From: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if self._order == self.PREORDER: if meth != None and meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) elif meth == None: self._iterate_over_others(node, *args) elif self._order == self.POSTORDER: if meth == None: self._iterate_over_others(node, *args) if meth != None: self._iterate_over_defaults(node, *args) meth(node, *args) To: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if meth is None: self._iterate_over_others(node, *args) else: if self._order == self.POSTORDER: self._iterate_over_defaults(node, *args) meth(node, *args) elif meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) else: pass # pre-order, and the method said to stop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 From noreply at sourceforge.net Wed Apr 25 00:45:36 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 15:45:36 -0700 Subject: [Patches] [ python-Patches-1706989 ] Implementation of @abstractmethod for PEP 3119 Message-ID: Patches item #1706989, was opened at 2007-04-24 18:45 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Implementation of @abstractmethod for PEP 3119 Initial Comment: This implements a new builtin, abstractmethod, which when used as a method decorator declares the method to be abstract, causing the class to be abstract (i.e. it cannot be instantiated). A subclass of an abstract class is still abstract unless it overrides all abstract base methods. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 From noreply at sourceforge.net Wed Apr 25 00:56:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 15:56:23 -0700 Subject: [Patches] [ python-Patches-1690201 ] Added support for custom readline functions Message-ID: Patches item #1690201, was opened at 2007-03-28 17:52 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ben Timby (btimby) Assigned to: Nobody/Anonymous (nobody) Summary: Added support for custom readline functions Initial Comment: This patch allows for binding a key or keyseq to a custom function named custom. A custom handler can be registered and will be called by readline when the registered sequence input. Example: rl.py -- import readline def handler(): print "need some help?" readline.set_custom_hook(handler) readline.parse_and_bind("\"?\": custom") text = raw_input("type ? for help# ") -- ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 18:56 Message: Logged In: YES user_id=764593 Originator: NO It looks as though you may have code from python 2.3.4 -- readline.c has been updated somewhat since then, particularly around threading. I'm not sure I fully understand what you're asking in step 3. It looks like on_hook(obj) will already call the python function well enough; you just need a way of keeping a reference to (and registering) a list of functions. The module uses (single) static variables for the hooks it predefines, and you followed that model; you could just add a list of objects. Instead of Then in set_custom_hook, instead of calling set_hook immediately, /* Not tested, you might want a dict instead, you should error check, you should pre-declare for C89, etc */ if (NULL == custom_hooks) custom_hooks=PyList_New(0); /* no initial elements */ PyList_Append(custom_hooks, args); /* add your custom func, etc to the list */ set_hook(" ", &(PyList_GET_ITEM(custom_hooks, PyList_GET_SIZE(custom_hooks))), *args); (Or you could just inline the checks from set_hook.) If you're feeling ambitious, you could fix set_hook itself to use a dictionary instead of specific pointers, or to get the name from the function.__name__, or to use a newer calling convention (As I understand it, the registration may really be METH_ONE, instead of METH_VARARGS.) ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-04-03 07:59 Message: Logged In: YES user_id=932679 Originator: YES Paul, thank you for clarifying. I agree with you completely. The reason I was unable to implement the API fully is the following problem (probably as you said due to the lack of dynamic features in C). 1. User defines python function. 2. User calls python readline.add_defun() 3. readline.add_defun() must whip up a C handler that will call the python function and in turn call rl_add_defun to register it. Problem being that multiple custom python functions registered with readline to a single C handler will have no way to determine which python function to call on invocation. 4. User calls readline.parse_and_bind() I have no clue how to do step 3, so I avoided it. If anyone has suggestions, ideas or examples, I am happy to do the work, I simply have not had to do something like this before in C. I don't even know what this would be called in order to google for it. The patch is certainly not as useful as the full API, however, I DID implement it to solve a problem, thus there is some use to it :-). I learned recently that I can maintain this feature as an extension (outside of) python, thus allowing me to continue to use it without depending on python maintainers to perceive usefulness. I will do this for the time being and if I am able to figure out item 3 I will resubmit the patch with a full implementation. ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-02 13:28 Message: Logged In: YES user_id=1740099 Originator: NO Hi Ben, I'm sorry if you thought my short reply suggested I hadn't looked at the patch properly. I'll try to clarify what I meant. Gnu readline provides an interface to custom functions: you declare a function that takes a 'count' and a 'key' and returns 0 or an error. You then call rl_add_defunc to declare it. You use the name you used in 'rl_add_defunc' in the readline init file to bind the command to a key. The patch uses a different interface: the user provides a function that takes no arguments. He declares it by calling 'set_custom_hook', and calls it by using the name 'custom' in the init file to bind it to a key. An interface consistent between python and readline would be: def say_hello(key, count): ... readline.add_defunc("say_hello", say_hello) readline.parse_and_bind('"?" : say_hello') (Perhaps the 'add_defun' is an artefact of lack of dynamic features of C and would be dropped from the python interface). The patch's use of a 'custom' hook is more complicated than using the function name directly in the init file, and provides less functionality (a single command, and removes the arguments that the c command would have). Accepting this patch will cause minor compatibility trouble in future if anyone works out a way round the admittedly difficult technical problems of doing things the 'right' way. Given the interface troubles, I'm not convinced enough of the usefulness of the patch. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-04-01 21:53 Message: Logged In: YES user_id=932679 Originator: YES If by adopt you mean adapt, that is what I have done. I simplified things to allow a single named function "custom" but it uses the gnu readline custom function interface (rl_add_defunc) . Have you even looked at the patch? ---------------------------------------------------------------------- Comment By: Paul Hankin (paulhankin) Date: 2007-04-01 17:47 Message: Logged In: YES user_id=1740099 Originator: NO Gnu readline supports custom functions in C - it would be better to adopt the gnu readline custom function interface to python, rather than using the 'custom' handler which smells of a hack. ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 19:01 Message: Logged In: YES user_id=932679 Originator: YES new Example: rl.py -- #!/usr/bin/python import readline def say_hello(): print print "this is my help text..." readline.on_new_line() readline.set_custom_hook(say_hello) readline.parse_and_bind("\"?\": custom") while True: data = raw_input("input# ") print "data: ", data -- ---------------------------------------------------------------------- Comment By: Ben Timby (btimby) Date: 2007-03-28 19:00 Message: Logged In: YES user_id=932679 Originator: YES File Added: python-2.3.4-readline_custom.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1690201&group_id=5470 From noreply at sourceforge.net Wed Apr 25 01:31:28 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 16:31:28 -0700 Subject: [Patches] [ python-Patches-1706989 ] Implementation of @abstractmethod for PEP 3119 Message-ID: Patches item #1706989, was opened at 2007-04-24 18:45 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Implementation of @abstractmethod for PEP 3119 Initial Comment: This implements a new builtin, abstractmethod, which when used as a method decorator declares the method to be abstract, causing the class to be abstract (i.e. it cannot be instantiated). A subclass of an abstract class is still abstract unless it overrides all abstract base methods. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-24 19:31 Message: Logged In: YES user_id=6380 Originator: YES Here's a version that compiles with C89 (GCC 2.96) and doesn't leak the 'fast' object. File Added: abstract.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 From noreply at sourceforge.net Wed Apr 25 02:08:41 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 17:08:41 -0700 Subject: [Patches] [ python-Patches-1221598 ] ftplib storbinary/storlines callback function Message-ID: Patches item #1221598, was opened at 2005-06-15 23:15 Message generated for change (Comment added) made by tcarroll You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1221598&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Phil Schwartz (phil_schwartz) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib storbinary/storlines callback function Initial Comment: The ftplib.FTP class currently allows the ability to provide a callback function for the retrbinary & retrlines methods to invoke. However, the corresponding storbinary and storlines methods do not. My ReleaseForge application currently uses the ftplib to send files (rather than retrieve) and it's helpful to know the percentage of the file that has been sent. For my app I subclassed ftplib.FTP and reimplemented storbinary to optionally accept a callback function as a parameter. However, I think this would be a useful method to include in the standard python distribution. I've attached a modified version of ftplib.py which adds this functionality. Unlike the retrX methods, the storX methods only invoke the callback function if it's not None so that it will not interfere with existing code. For completeness, the retrX methods invoke print_line if a callback function is not provided and doing so for the storX functions may be undesired. ---------------------------------------------------------------------- Comment By: Terry Carroll (tcarroll) Date: 2007-04-25 00:08 Message: Logged In: YES user_id=755895 Originator: NO I'd like to voice some support for this patch, if that's not out of line. I've used it myself in at least one app, and it would be nice to have it as a standard feature in 2.6. It seems odd that the callback feature is in the two retrieval methods, but not the store methods. Ruby, for example, supports it in both. Not that Python has to do everything Ruby does, but there doesn't seem to be any obstacle to adding this support. ---------------------------------------------------------------------- Comment By: Phil Schwartz (phil_schwartz) Date: 2006-07-03 18:09 Message: Logged In: YES user_id=420501 Attached patch of ftplib.py (of Python 2.4.3) ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-03-28 07:52 Message: Logged In: YES user_id=849994 Please provide your patch as a diff between the original ftplib.py and your version. It greatly helps reviewers. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1221598&group_id=5470 From noreply at sourceforge.net Wed Apr 25 05:14:24 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 20:14:24 -0700 Subject: [Patches] [ python-Patches-1706039 ] Added clearerr() to clear EOF state Message-ID: Patches item #1706039, was opened at 2007-04-23 13:21 Message generated for change (Comment added) made by belopolsky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: jos (josm) Assigned to: Nobody/Anonymous (nobody) Summary: Added clearerr() to clear EOF state Initial Comment: This patch is to fix bug 1523853. Now file.read() works as written in the doc, which says "An empty string is returned when EOF is encountered immediately." Before this fix, An empty string was returned even when the last read() encountered EOF. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-24 23:14 Message: Logged In: YES user_id=835142 Originator: NO I have just verified that this issue affects readlines and readinto methods on Mac OS X as well. This tells me that rather than patching all the individual methods, maybe Py_UniversalNewlineFread should be fixed instead. I believe that the current situation where a function that looks like Python API reports errors by setting flags on a FILE object instead of settin the an exception is against python's design. I suggest changing Py_UniversalNewlineFgets so that if fread returns 0, it checks whether that was due to EOF or an error and sets an exception in a case of an error and always calls clearerr. ---------------------------------------------------------------------- Comment By: jos (josm) Date: 2007-04-24 13:15 Message: Logged In: YES user_id=1776568 Originator: YES I confirmed that fread can read data from fp in EOF state on Linux. On OS X, freading from fp in EOF state doesn't work, just return 0. I tested this behavior using the following code. ---------------------------- #include #include main(void) { FILE *fp1, *fp2; int n; char buf[128]; fp1 = fopen("spam.txt", "w+"); fputs("foo", fp1); fseek(fp1, 0L, SEEK_END); if (fread(buf, 1, sizeof(buf), fp1) == 0) fprintf(stderr, "first fread failed\n"); fp2 = fopen("spam.txt", "a+"); fputs("bar", fp2); fclose(fp2); if (feof(fp1)) fprintf(stderr, "after appending some text, fp1 is still eof\n"); if (fread(buf, 1, sizeof(buf), fp1) == 0) fprintf(stderr, "second fread failed\n"); printf("buf: %s\n", buf); fclose(fp1); return 0; } ---------------------------- ============= On Linux ============= first fread failed after appending some text, fp1 is still eof buf: bar ============= On OS X ============= first fread failed after appending some text, fp1 is still eof second fread failed buf: Anyway, I think it's safe and preferable to clear EOF indicator in this case. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-24 00:28 Message: Logged In: YES user_id=835142 Originator: NO Hmm. It looks like glibc's fread will attempt reading even if EOF flag is already set. I've reproduced the problem on OS X 10.4.9 and Python 2.6a0 (trunk:54926M). This patch fixes it. ---------------------------------------------------------------------- Comment By: jos (josm) Date: 2007-04-23 19:27 Message: Logged In: YES user_id=1776568 Originator: YES I can easily reproduce this problem on OS X 10.4.9 using Python 2.4.3 and Python 2.5. Here's the code ################# import os, sys filename = "spam" f = open(filename, "w+") f.seek(0, 2) line = f.read() # EOF # Writing the same file using another fd f2 = open(filename, "a+") f2.write("Spam") f2.flush() f2.close() statinfo = os.stat(filename) print "file size: %d" % statinfo.st_size print "position : %d" % f.tell() line = f.read() # read() returns emtpy!! readlines?() works ok print "line : [%s]" % line ################# On my machine, it outputs the following ### file size: 4 position : 0 line : [] ### And just now I found that on ubuntu on the same machine (vmware), it works collect. ### file size: 4 position : 0 line : [Spam] ### My patched version python outputs the same result as above on OS X. We need clearerr() there, too because clearerr()'s job is not only only clearing error indicators but EOF indicators also. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-23 16:22 Message: Logged In: YES user_id=835142 Originator: NO I was not able to reproduce the problem. I tried to read through the end of the file, append to the file from an external process and read again. It worked as expected: more data was read. The proposed patch seems to be redundant: once ferror(f->f_fp) is known to be 0, clearerr will do nothing and in the alternative branch clearerr is already there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 From noreply at sourceforge.net Wed Apr 25 05:55:28 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 20:55:28 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 03:39 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-24 23:55 Message: Logged In: YES user_id=18139 Originator: YES OK, the updated patch has the suggested changes (Doc for EntityReference; eliminated "pass" and added comment in Node.__setattr__). File Added: minidom-level-1-compliance.2.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 07:53 Message: Logged In: YES user_id=18139 Originator: YES Neal, thanks for looking at this. Size: Yeah. :-\ I don't know what to do about this. There were a lot of bugs, so there are a lot of diffs. :( Whitespace: I checked, and the extra whitespace is definitely in trunk's Lib/test/test_minidom.py; a recent change added it. This patch removes it. (I could revert the whitespace anyway, just to reduce the size of the patch. Let me know if I should do this.) Doc: Oops! I will fix this today and upload a new patch. pass: Thanks, I'll change as suggested. (Just to explain myself-- I used "if name == 'nodeValue': pass; else: ..." because 'nodeValue' is the special case, and the special behavior is "do nothing". But I'm not attached to that wording. I'll change it.) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 02:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 09:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Wed Apr 25 07:35:03 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 22:35:03 -0700 Subject: [Patches] [ python-Patches-1706989 ] Implementation of @abstractmethod for PEP 3119 Message-ID: Patches item #1706989, was opened at 2007-04-24 15:45 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Implementation of @abstractmethod for PEP 3119 Initial Comment: This implements a new builtin, abstractmethod, which when used as a method decorator declares the method to be abstract, causing the class to be abstract (i.e. it cannot be instantiated). A subclass of an abstract class is still abstract unless it overrides all abstract base methods. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 22:35 Message: Logged In: YES user_id=33168 Originator: NO Perhaps this is a better question for the PEP rather than the impl, but can attributes be abstract? class Foo: abstract_override_me = ??? If so, then __isabstractmethod__ might be better named as: __isabstract__. I think this might work: class Abstract: __isabstractmethod__ = True class Foo: abstract_override_me = Abstract() Do you want arbitrary objects to be able to declare their abstractness or should the impl also check that an attribute is callable? check_new_abstracts() should return a Py_ssize_t since it returns the size of a container (set). The return value is already captured in a Py_ssize_t, so it's just the signature (and prototype) that should change. PySet_Add()s return value isn't checked in check_new_abstracts(). It might be nice to factor out the common code between the two new functions into a static helper function. That would get rid of the PySet_Add problem. By calling: PyObject_GetAttrString(meth, "__isabstractmethod__"), that means a new string object is allocated and then thrown away with each call. This could be improved by creating an interned string for "__isabstractmethod__". (I realize this is only when types are created which shouldn't be too often.) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-24 16:31 Message: Logged In: YES user_id=6380 Originator: YES Here's a version that compiles with C89 (GCC 2.96) and doesn't leak the 'fast' object. File Added: abstract.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 From noreply at sourceforge.net Wed Apr 25 07:39:45 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 22:39:45 -0700 Subject: [Patches] [ python-Patches-1221598 ] ftplib storbinary/storlines callback function Message-ID: Patches item #1221598, was opened at 2005-06-15 16:15 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1221598&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Phil Schwartz (phil_schwartz) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib storbinary/storlines callback function Initial Comment: The ftplib.FTP class currently allows the ability to provide a callback function for the retrbinary & retrlines methods to invoke. However, the corresponding storbinary and storlines methods do not. My ReleaseForge application currently uses the ftplib to send files (rather than retrieve) and it's helpful to know the percentage of the file that has been sent. For my app I subclassed ftplib.FTP and reimplemented storbinary to optionally accept a callback function as a parameter. However, I think this would be a useful method to include in the standard python distribution. I've attached a modified version of ftplib.py which adds this functionality. Unlike the retrX methods, the storX methods only invoke the callback function if it's not None so that it will not interfere with existing code. For completeness, the retrX methods invoke print_line if a callback function is not provided and doing so for the storX functions may be undesired. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 22:39 Message: Logged In: YES user_id=33168 Originator: NO Terry, your comment is definitely *not* out of line! The more people comment on patches the easier it is for us to make a decision. It would be great to comment on other patches or bugs that affect you. Thanks. ---------------------------------------------------------------------- Comment By: Terry Carroll (tcarroll) Date: 2007-04-24 17:08 Message: Logged In: YES user_id=755895 Originator: NO I'd like to voice some support for this patch, if that's not out of line. I've used it myself in at least one app, and it would be nice to have it as a standard feature in 2.6. It seems odd that the callback feature is in the two retrieval methods, but not the store methods. Ruby, for example, supports it in both. Not that Python has to do everything Ruby does, but there doesn't seem to be any obstacle to adding this support. ---------------------------------------------------------------------- Comment By: Phil Schwartz (phil_schwartz) Date: 2006-07-03 11:09 Message: Logged In: YES user_id=420501 Attached patch of ftplib.py (of Python 2.4.3) ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-03-27 23:52 Message: Logged In: YES user_id=849994 Please provide your patch as a diff between the original ftplib.py and your version. It greatly helps reviewers. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1221598&group_id=5470 From noreply at sourceforge.net Wed Apr 25 08:03:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 23:03:54 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 19:49 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-25 06:03 Message: Logged In: YES user_id=849994 Originator: NO So this can be closed? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-20 20:15 Message: Logged In: YES user_id=642936 Originator: NO I checked in a fix in the release25-maint branch (rev. 54889), so it will show up in the 2.5.2 release. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 14:07 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 13:57 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-07 09:33 Message: Logged In: YES user_id=642936 Originator: NO Thank you for making clear that this is a regression. This is a completely different thing and makes the previous discussion obsolete... Unfortunately, this will not go into 2.5.1, we're too late. But I will come up with a reasonable fix and testcases for 2.5.2. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 22:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 21:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 18:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 18:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 10:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 10:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 09:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 09:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 20:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Wed Apr 25 08:04:22 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 23:04:22 -0700 Subject: [Patches] [ python-Patches-1698768 ] Update to Macintosh library documentation Message-ID: Patches item #1698768, was opened at 2007-04-11 20:40 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: Accepted Priority: 5 Private: No Submitted By: Kevin Walzer (wordtech) >Assigned to: Georg Brandl (gbrandl) Summary: Update to Macintosh library documentation Initial Comment: The attached file updates section 1 of the Macintosh library documentation, which covers getting started--installing, configuring, and learning Python on the Mac. The file updates the section to reflect changes in the Mac version of Python since 2.4 (switching to IDLE as the standard IDE, supporting universal binaries, etc.) and adds sections on application deployment with py2app, GUI toolkits, application scripting, and more. Much of the current documentation in this section is obsolete and therefore inaccurate. The updated documentation is a simple text file: I am not an expert with LaTeX. I understand the Python documentation team can handle the LaTeX formatting. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-25 06:04 Message: Logged In: YES user_id=849994 Originator: NO I'll handle that. ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2007-04-20 06:02 Message: Logged In: YES user_id=580910 Originator: NO This looks great. I'm definitely in favor of using this document instead of the current version. IMHO this change should also be backported to the 2.5 branch of the documention. Brett: I'm assigning this back to you because my LaTeX-fu is seriously rusted. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2007-04-11 23:30 Message: Logged In: YES user_id=357491 Originator: NO Thanks for the work, Kevin! Ronald, can you have a look at this and approve or reject? Once that is done we can work on converting it to LaTeX. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 From noreply at sourceforge.net Wed Apr 25 08:15:28 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 23:15:28 -0700 Subject: [Patches] [ python-Patches-1703379 ] Refactor test_frozen.py to use unittest. Message-ID: Patches item #1703379, was opened at 2007-04-18 21:49 Message generated for change (Comment added) made by jseutter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jerry Seutter (jseutter) Assigned to: Neal Norwitz (nnorwitz) Summary: Refactor test_frozen.py to use unittest. Initial Comment: This patch refactors test_frozen.py to use unittest. When this patch is applied, Lib/test/output/test_frozen can be removed. I am not sure if the last test case is named appropriately. I named it testImportNonexistentFunction because I believe the code is trying to import a function __phello__.foo, which does not exist. I may be mistaken. ---------------------------------------------------------------------- >Comment By: Jerry Seutter (jseutter) Date: 2007-04-25 00:15 Message: Logged In: YES user_id=1727609 Originator: YES Ahh, that makes sense. Thanks Neal! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 21:37 Message: Logged In: YES user_id=33168 Originator: NO Thanks for updating the patch, I'll try to take a look at it soon (within a week or so). If anyone else wants to take a look at this, feel free. > Why doesn't regrtest give an error if it doesn't run tests? regrtest can't know the difference between an old style test which executes on import and a new style test which has a test_main. This is why I want to get rid of all the old style tests (the ones that execute by side-effect of import). Then we can have regrtest complain if there is no test_main method just as you suggest. ---------------------------------------------------------------------- Comment By: Jerry Seutter (jseutter) Date: 2007-04-23 18:39 Message: Logged In: YES user_id=1727609 Originator: YES Thanks for the input, guys. See version 2 of the patch which should resolve these issues. Question: Why doesn't regrtest give an error if it doesn't run tests? File Added: refactor_test_frozen.py.patch ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2007-04-19 00:18 Message: Logged In: YES user_id=580910 Originator: NO As Neal wrote sys.platform == 'mac' is of OS9, which isn't supported anymore by Python (or Apple for that matter). The special-case for platform == 'mac' should be removed altoghether and not replaced by a check for OSX because OSX is basically just another unix platform and should therefore support freezing. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-19 00:08 Message: Logged In: YES user_id=33168 Originator: NO Thanks for the patch! It looks pretty good, but there is one problem I see. There should be a test_main() function for this to work from regrtest. For example, take a look in Lib/test/test_bool.py. Also, I don't think you need to seek(0) before getting the value from sys.stdout (ie, cStringIO instance). I don't think you need to support sys.platform == 'mac'. That was for Mac OS 9 which is no longer supported (see PEP 11). Mac OS 10 has sys.platform == 'darwin'. If you're interested, it would be good to produce a patch to remove all (10+) occurrences of comparing sys.platform with mac. I'm assigning this to Ronald just so he can verify the Mac OS 9 issue. Ronald, please comment and then unassign or assign to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1703379&group_id=5470 From noreply at sourceforge.net Wed Apr 25 08:26:09 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 23:26:09 -0700 Subject: [Patches] [ python-Patches-1698768 ] Update to Macintosh library documentation Message-ID: Patches item #1698768, was opened at 2007-04-11 20:40 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None >Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Kevin Walzer (wordtech) Assigned to: Georg Brandl (gbrandl) Summary: Update to Macintosh library documentation Initial Comment: The attached file updates section 1 of the Macintosh library documentation, which covers getting started--installing, configuring, and learning Python on the Mac. The file updates the section to reflect changes in the Mac version of Python since 2.4 (switching to IDLE as the standard IDE, supporting universal binaries, etc.) and adds sections on application deployment with py2app, GUI toolkits, application scripting, and more. Much of the current documentation in this section is obsolete and therefore inaccurate. The updated documentation is a simple text file: I am not an expert with LaTeX. I understand the Python documentation team can handle the LaTeX formatting. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2007-04-25 06:26 Message: Logged In: YES user_id=849994 Originator: NO Committed rev. 54949, 54950 (2.5). ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-04-25 06:04 Message: Logged In: YES user_id=849994 Originator: NO I'll handle that. ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2007-04-20 06:02 Message: Logged In: YES user_id=580910 Originator: NO This looks great. I'm definitely in favor of using this document instead of the current version. IMHO this change should also be backported to the 2.5 branch of the documention. Brett: I'm assigning this back to you because my LaTeX-fu is seriously rusted. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2007-04-11 23:30 Message: Logged In: YES user_id=357491 Originator: NO Thanks for the work, Kevin! Ronald, can you have a look at this and approve or reject? Once that is done we can work on converting it to LaTeX. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1698768&group_id=5470 From noreply at sourceforge.net Wed Apr 25 08:26:54 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 24 Apr 2007 23:26:54 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 00:39 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 23:26 Message: Logged In: YES user_id=33168 Originator: NO Jason, don't worry about the size, I was mostly complaining about the whitespace diffs. I know Tim hadn't done a normalization step for a while. I just did it and am running the tests now. Damn there are a lot! :-( The good news is that your patch will go down. The bad news for you is that you are going to have to regenerate it and there will probably be conflicts. I'm hoping Martin will review it as he's quite knowledgeable about XML. I doubt there is any more that I can contribute to this patch other than to say it needs some expert eyes on it, but generally looks good. The tests should be done within 15 minutes or so. Then I'll check in whitespace normalization. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 20:55 Message: Logged In: YES user_id=18139 Originator: YES OK, the updated patch has the suggested changes (Doc for EntityReference; eliminated "pass" and added comment in Node.__setattr__). File Added: minidom-level-1-compliance.2.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 04:53 Message: Logged In: YES user_id=18139 Originator: YES Neal, thanks for looking at this. Size: Yeah. :-\ I don't know what to do about this. There were a lot of bugs, so there are a lot of diffs. :( Whitespace: I checked, and the extra whitespace is definitely in trunk's Lib/test/test_minidom.py; a recent change added it. This patch removes it. (I could revert the whitespace anyway, just to reduce the size of the patch. Let me know if I should do this.) Doc: Oops! I will fix this today and upload a new patch. pass: Thanks, I'll change as suggested. (Just to explain myself-- I used "if name == 'nodeValue': pass; else: ..." because 'nodeValue' is the special case, and the special behavior is "do nothing". But I'm not attached to that wording. I'll change it.) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-23 23:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 06:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Wed Apr 25 09:04:17 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 00:04:17 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed Resolution: None Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-04-25 08:03 Message: Logged In: YES user_id=849994 Originator: NO So this can be closed? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-20 22:15 Message: Logged In: YES user_id=642936 Originator: NO I checked in a fix in the release25-maint branch (rev. 54889), so it will show up in the 2.5.2 release. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 16:07 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 15:57 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-07 11:33 Message: Logged In: YES user_id=642936 Originator: NO Thank you for making clear that this is a regression. This is a completely different thing and makes the previous discussion obsolete... Unfortunately, this will not go into 2.5.1, we're too late. But I will come up with a reasonable fix and testcases for 2.5.2. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 00:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 23:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Wed Apr 25 09:07:17 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 00:07:17 -0700 Subject: [Patches] [ python-Patches-1695229 ] tarfile bug when opening a file directly Message-ID: Patches item #1695229, was opened at 2007-04-05 21:49 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Arve Knudsen (arve_knudsen) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile bug when opening a file directly Initial Comment: The tarfile.open function supports two ways of opening a file, either indirectly by specifying its name (parameter 'name') or by passing an open file object (parameter 'fileobj'). If one only passes a file object however an exception is raised, since on line 1047 TarFile.__init__ passes the filename to os.path.abspath, regardless of whether it is None or not. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2007-04-25 08:03 Message: Logged In: YES user_id=849994 Originator: NO So this can be closed? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-20 22:15 Message: Logged In: YES user_id=642936 Originator: NO I checked in a fix in the release25-maint branch (rev. 54889), so it will show up in the 2.5.2 release. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 16:07 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 15:57 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-07 11:33 Message: Logged In: YES user_id=642936 Originator: NO Thank you for making clear that this is a regression. This is a completely different thing and makes the previous discussion obsolete... Unfortunately, this will not go into 2.5.1, we're too late. But I will come up with a reasonable fix and testcases for 2.5.2. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-07 00:35 Message: Logged In: YES user_id=1522083 Originator: YES Lars: AFAICT this is a regression in 2.5, since my code worked without problems with 2.4. Also, how can it be a feature that TarFile.__init__ doesn't support its own default value for the 'name' parameter? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 23:41 Message: Logged In: YES user_id=984087 Originator: NO I completely agree that it can be worked around very simply and so may not merit a fix in the current release. May be, a simple doc change would suffice. But I fail to understand how fixing this issue in 2.5 would break some existing code. There will be no current code that uses open without "name" (as that results in exception) and allowing that usage should not break any code. No? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 22:14 Message: Logged In: YES user_id=642936 Originator: NO I admit that the docs for 2.5 lack explicitness in this aspect. It is not explicitly said that fileobj may be passed without a name argument. However, in the method signature name is listed as optional. If no name argument is passed I would say behaviour is *undefined* atm. I may be too pedantic, but: 1. It is perfectly simple to work around this problem. 2. It is just a question of code-niceness. 3. It is already fixed for the next major release. I do not like to take the risk to break someone's code with a bugfix release. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-06 20:55 Message: Logged In: YES user_id=984087 Originator: NO Hi, I may be missing something here but the 2.5 document lists "name" parameter as optional. So if not using it results in an exception, it should be a bug. Right? If there is a mention in the document that "name" must be passed (directly or indirectly), can you please point that out? As I said, I might have missed it. If this is accepted as a bug, then it alone can be fixed instead of back porting entire 54335 changes?. Of course, I don't know how easy it is to fix this issue alone and if is closely intertwined with those changes, then it doesn't make sense to fix this issue separately in 2.5. Thanks, Raghu. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 20:03 Message: Logged In: YES user_id=642936 Originator: NO I'm afraid not. This is a new feature not a bugfix. The 2.5 behaviour is not a bug, it is documented. Backporting the new behaviour would make it necessary to change the documentation, too. Because of Python release policy and 2.5.1 being a bugfix release, this cannot go in, I'm afraid. You must wait for the 2.6 release and work around this in the meantime. Sorry. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 12:23 Message: Logged In: YES user_id=1522083 Originator: YES Can we expect a fixed 2.5 release soon? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2007-04-06 12:09 Message: Logged In: YES user_id=642936 Originator: NO This was fixed with rev. 54335 (trunk) as part of a larger changeset. ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:59 Message: Logged In: YES user_id=1522083 Originator: YES File Added: tarfile.patch ---------------------------------------------------------------------- Comment By: Arve Knudsen (arve_knudsen) Date: 2007-04-06 11:52 Message: Logged In: YES user_id=1522083 Originator: YES I tested on Ubuntu Linux. Viewing the trunk revision of tarfile.py, I can see the bug remains. That is, TarFile.__init__ has a default value of None for the name parameter, but still it passes this parameter directly to os.path.abspath which does not expect a None value. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-05 22:26 Message: Logged In: YES user_id=984087 Originator: NO When I tested the following code with python 2.5 on linux, I got an exception: ------------------- Python 2.5 (r25:51908, Jan 24 2007, 12:48:15) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() Traceback (most recent call last): File "", line 1, in File "/usr/local/lib/python2.5/tarfile.py", line 1019, in open return func(name, "r", fileobj) File "/usr/local/lib/python2.5/tarfile.py", line 1077, in gzopen pre, ext = os.path.splitext(name) File "/usr/local/lib/python2.5/posixpath.py", line 92, in splitext i = p.rfind('.') AttributeError: 'NoneType' object has no attribute 'rfind' >>> --------------------- However, the test with latest python (built from latest code in svn) passes. ---------------------- Python 2.6a0 (trunk:54698M, Apr 5 2007, 16:18:31) [GCC 4.1.0 (SUSE Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import tarfile >>> f = open("test.tar") >>> tarfile.open(fileobj=f).getnames() ['wikipediafs-0.2', 'wikipediafs-0.2/AUTHORS', 'wikipediafs-0.2/ChangeLog', 'wikipediafs-0.2/COPYING', 'wikipediafs-0.2/COPYRIGHT', 'wikipediafs-0.2/VERSION', 'wikipediafs-0.2/README', 'wikipediafs-0.2/setup.py', 'wikipediafs-0.2/site', 'wikipediafs-0.2/site/index.htm', 'wikipediafs-0.2/site/index_fr.htm', 'wikipediafs-0.2/src', 'wikipediafs-0.2/src/wikipediafs', 'wikipediafs-0.2/src/wikipediafs/WikipediaFS.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticleList.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaUser.py', 'wikipediafs-0.2/src/wikipediafs/WikipediaArticle.py', 'wikipediafs-0.2/src/wikipediafs/__init__.py', 'wikipediafs-0.2/src/wikipediafs/logger.py', 'wikipediafs-0.2/src/wikipediafs/version.py', 'wikipediafs-0.2/src/mount.wikipediafs', 'wikipediafs-0.2/TODO'] >> --------------------------- Which platform did you test on? You may want to give the latest version a try as well. Thanks, Raghu. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1695229&group_id=5470 From noreply at sourceforge.net Wed Apr 25 10:33:05 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 01:33:05 -0700 Subject: [Patches] [ python-Patches-1114345 ] Add SSL certificate validation Message-ID: Patches item #1114345, was opened at 2005-02-02 00:04 Message generated for change (Comment added) made by vila-sf You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1114345&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: James Eagan (noonian) Assigned to: Nobody/Anonymous (nobody) Summary: Add SSL certificate validation Initial Comment: One line summary: adds certificate validation to the SSL module and programmer-level hooks to control how and whether certificate validation is performed. Details: The current SSL implementation in python goes through the motions of negotiating an SSL connection, but never validates the certificates exchanged. This is like going through the motions of checking someone's photo id, but never making sure the picture matches the person you're talking to. This patch fixes that. This patch adds 3 module-level variables to the socket module, which get exposed iff ssl is built in. These variables (ssl_ca_file, ssl_ca_path, and ssl_verify_level) provide programmer-level access to the certificate authorities database and to control what level of certificate verification is performed (by default, none, as is the current behavior). If certificate verification is enabled, then one of the two certificate authority parameters must be set to a valid certificate authority database or all certificate verification operations will fail. I have an example certificate authority database (extracted from the Java keystore) that I can provide, but I'm not sure how to contribute that through the patch mechanism. Cheers! James Eagan ---------------------------------------------------------------------- Comment By: vila-sf (vila-sf) Date: 2007-04-25 10:33 Message: Logged In: YES user_id=1584517 Originator: NO Like nagle, I have a need for this. But the updated patch is now returning a 404 :-( Since we are at still three interested by that patch, what can we do ? ---------------------------------------------------------------------- Comment By: James Eagan (noonian) Date: 2006-11-30 22:09 Message: Logged In: YES user_id=31389 Originator: YES I'd be happy to make the changes l?wis suggested, but it will be quite a while before I can find the necessary time. If anyone else can update the docs and tests, please let me know! ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-30 21:50 Message: Logged In: YES user_id=21627 Originator: NO The patch is not integrated because nobody had the time to review it; this, in turn, did not happen because we lack reviewers. A quick review reveals that the patch is incomplete: it does not provide changes to the documentation (which it needs to, because it introduces a new feature). The patch also includes no changes to the test suite. ---------------------------------------------------------------------- Comment By: John Nagle (nagle) Date: 2006-11-30 21:13 Message: Logged In: YES user_id=5571 Originator: NO This has been pending for a few months, and there's a fix, but it's not in yet. What's going on? I just had Python accept a totally bogus certificate from "www.amaison.co.uk". The certificate contents are C = -- ST = SomeState L = SomeCity O = SomeOrganization OU = SomeOrganizationalUnit CN = localhost.localdomain emailAddress = root at localhost.localdomain Issuer identity: C = -- ST = SomeState L = SomeCity O = SomeOrganization OU = SomeOrganizationalUnit CN = localhost.localdomain emailAddress = root at localhost.localdomain Python is perfectly happy with that. Which is embarassing. ---------------------------------------------------------------------- Comment By: Gustavo J. A. M. Carneiro (gustavo) Date: 2006-11-09 16:20 Message: Logged In: YES user_id=908 > This patch adds 3 module-level variables to the socket module, which get exposed iff ssl is built in. These variables (ssl_ca_file, ssl_ca_path, and ssl_verify_level) provide programmer-level access to the certificate authorities database and to control what level of certificate verification is performed (by default, none, as is the current behavior). Are you sure it's a good idea to have this kind of 'global' control over certification authorities? Global configurations are handy at first, but they come back and bite us when we least expect it... ---------------------------------------------------------------------- Comment By: James Eagan (noonian) Date: 2006-11-09 15:43 Message: Logged In: YES user_id=31389 Nagle: I haven't heard anything from anyone besides you and jbowes abou this patch here or on the python-dev list, and I haven't had time to follow up. You might have more success via the email list. (Or, if any of the python maintainers is reading this, do you have any suggestions to make this patch more attractive?) ---------------------------------------------------------------------- Comment By: John Nagle (nagle) Date: 2006-11-09 05:14 Message: Logged In: YES user_id=5571 What's the status of this? Is it going in? I have a need for it. Thanks. ---------------------------------------------------------------------- Comment By: James Bowes (jbowes) Date: 2006-06-21 21:43 Message: Logged In: YES user_id=1543815 I put together an updated version of this patch against svn trunk as of June 21, 2006. I also added some additional documentation to the .tex file. Maybe someone with sufficient privilidges (or James, if you're still out there) could attach the updated patch here? the updated patch is at: http://www.dangerouslyinc.com/~bowes/ssl_ca.diff Regards, James Bowes ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1114345&group_id=5470 From noreply at sourceforge.net Wed Apr 25 10:35:50 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 01:35:50 -0700 Subject: [Patches] [ python-Patches-1114345 ] Add SSL certificate validation Message-ID: Patches item #1114345, was opened at 2005-02-02 00:04 Message generated for change (Comment added) made by vila-sf You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1114345&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: James Eagan (noonian) Assigned to: Nobody/Anonymous (nobody) Summary: Add SSL certificate validation Initial Comment: One line summary: adds certificate validation to the SSL module and programmer-level hooks to control how and whether certificate validation is performed. Details: The current SSL implementation in python goes through the motions of negotiating an SSL connection, but never validates the certificates exchanged. This is like going through the motions of checking someone's photo id, but never making sure the picture matches the person you're talking to. This patch fixes that. This patch adds 3 module-level variables to the socket module, which get exposed iff ssl is built in. These variables (ssl_ca_file, ssl_ca_path, and ssl_verify_level) provide programmer-level access to the certificate authorities database and to control what level of certificate verification is performed (by default, none, as is the current behavior). If certificate verification is enabled, then one of the two certificate authority parameters must be set to a valid certificate authority database or all certificate verification operations will fail. I have an example certificate authority database (extracted from the Java keystore) that I can provide, but I'm not sure how to contribute that through the patch mechanism. Cheers! James Eagan ---------------------------------------------------------------------- Comment By: vila-sf (vila-sf) Date: 2007-04-25 10:35 Message: Logged In: YES user_id=1584517 Originator: NO Like nagle, I have a need for this. But the updated patch is now returning a 404 :-( Since we are at still three interested by that patch, what can we do ? ---------------------------------------------------------------------- Comment By: vila-sf (vila-sf) Date: 2007-04-25 10:33 Message: Logged In: YES user_id=1584517 Originator: NO Like nagle, I have a need for this. But the updated patch is now returning a 404 :-( Since we are at still three interested by that patch, what can we do ? ---------------------------------------------------------------------- Comment By: James Eagan (noonian) Date: 2006-11-30 22:09 Message: Logged In: YES user_id=31389 Originator: YES I'd be happy to make the changes l?wis suggested, but it will be quite a while before I can find the necessary time. If anyone else can update the docs and tests, please let me know! ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-30 21:50 Message: Logged In: YES user_id=21627 Originator: NO The patch is not integrated because nobody had the time to review it; this, in turn, did not happen because we lack reviewers. A quick review reveals that the patch is incomplete: it does not provide changes to the documentation (which it needs to, because it introduces a new feature). The patch also includes no changes to the test suite. ---------------------------------------------------------------------- Comment By: John Nagle (nagle) Date: 2006-11-30 21:13 Message: Logged In: YES user_id=5571 Originator: NO This has been pending for a few months, and there's a fix, but it's not in yet. What's going on? I just had Python accept a totally bogus certificate from "www.amaison.co.uk". The certificate contents are C = -- ST = SomeState L = SomeCity O = SomeOrganization OU = SomeOrganizationalUnit CN = localhost.localdomain emailAddress = root at localhost.localdomain Issuer identity: C = -- ST = SomeState L = SomeCity O = SomeOrganization OU = SomeOrganizationalUnit CN = localhost.localdomain emailAddress = root at localhost.localdomain Python is perfectly happy with that. Which is embarassing. ---------------------------------------------------------------------- Comment By: Gustavo J. A. M. Carneiro (gustavo) Date: 2006-11-09 16:20 Message: Logged In: YES user_id=908 > This patch adds 3 module-level variables to the socket module, which get exposed iff ssl is built in. These variables (ssl_ca_file, ssl_ca_path, and ssl_verify_level) provide programmer-level access to the certificate authorities database and to control what level of certificate verification is performed (by default, none, as is the current behavior). Are you sure it's a good idea to have this kind of 'global' control over certification authorities? Global configurations are handy at first, but they come back and bite us when we least expect it... ---------------------------------------------------------------------- Comment By: James Eagan (noonian) Date: 2006-11-09 15:43 Message: Logged In: YES user_id=31389 Nagle: I haven't heard anything from anyone besides you and jbowes abou this patch here or on the python-dev list, and I haven't had time to follow up. You might have more success via the email list. (Or, if any of the python maintainers is reading this, do you have any suggestions to make this patch more attractive?) ---------------------------------------------------------------------- Comment By: John Nagle (nagle) Date: 2006-11-09 05:14 Message: Logged In: YES user_id=5571 What's the status of this? Is it going in? I have a need for it. Thanks. ---------------------------------------------------------------------- Comment By: James Bowes (jbowes) Date: 2006-06-21 21:43 Message: Logged In: YES user_id=1543815 I put together an updated version of this patch against svn trunk as of June 21, 2006. I also added some additional documentation to the .tex file. Maybe someone with sufficient privilidges (or James, if you're still out there) could attach the updated patch here? the updated patch is at: http://www.dangerouslyinc.com/~bowes/ssl_ca.diff Regards, James Bowes ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1114345&group_id=5470 From noreply at sourceforge.net Wed Apr 25 14:02:35 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 05:02:35 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 03:39 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-25 08:02 Message: Logged In: YES user_id=18139 Originator: YES File Added: minidom-level-1-compliance.3.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-25 02:26 Message: Logged In: YES user_id=33168 Originator: NO Jason, don't worry about the size, I was mostly complaining about the whitespace diffs. I know Tim hadn't done a normalization step for a while. I just did it and am running the tests now. Damn there are a lot! :-( The good news is that your patch will go down. The bad news for you is that you are going to have to regenerate it and there will probably be conflicts. I'm hoping Martin will review it as he's quite knowledgeable about XML. I doubt there is any more that I can contribute to this patch other than to say it needs some expert eyes on it, but generally looks good. The tests should be done within 15 minutes or so. Then I'll check in whitespace normalization. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 23:55 Message: Logged In: YES user_id=18139 Originator: YES OK, the updated patch has the suggested changes (Doc for EntityReference; eliminated "pass" and added comment in Node.__setattr__). File Added: minidom-level-1-compliance.2.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 07:53 Message: Logged In: YES user_id=18139 Originator: YES Neal, thanks for looking at this. Size: Yeah. :-\ I don't know what to do about this. There were a lot of bugs, so there are a lot of diffs. :( Whitespace: I checked, and the extra whitespace is definitely in trunk's Lib/test/test_minidom.py; a recent change added it. This patch removes it. (I could revert the whitespace anyway, just to reduce the size of the patch. Let me know if I should do this.) Doc: Oops! I will fix this today and upload a new patch. pass: Thanks, I'll change as suggested. (Just to explain myself-- I used "if name == 'nodeValue': pass; else: ..." because 'nodeValue' is the special case, and the special behavior is "do nothing". But I'm not attached to that wording. I'll change it.) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 02:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 09:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Wed Apr 25 18:44:52 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 09:44:52 -0700 Subject: [Patches] [ python-Patches-1114345 ] Add SSL certificate validation Message-ID: Patches item #1114345, was opened at 2005-02-01 23:04 Message generated for change (Comment added) made by nagle You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1114345&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: James Eagan (noonian) Assigned to: Nobody/Anonymous (nobody) Summary: Add SSL certificate validation Initial Comment: One line summary: adds certificate validation to the SSL module and programmer-level hooks to control how and whether certificate validation is performed. Details: The current SSL implementation in python goes through the motions of negotiating an SSL connection, but never validates the certificates exchanged. This is like going through the motions of checking someone's photo id, but never making sure the picture matches the person you're talking to. This patch fixes that. This patch adds 3 module-level variables to the socket module, which get exposed iff ssl is built in. These variables (ssl_ca_file, ssl_ca_path, and ssl_verify_level) provide programmer-level access to the certificate authorities database and to control what level of certificate verification is performed (by default, none, as is the current behavior). If certificate verification is enabled, then one of the two certificate authority parameters must be set to a valid certificate authority database or all certificate verification operations will fail. I have an example certificate authority database (extracted from the Java keystore) that I can provide, but I'm not sure how to contribute that through the patch mechanism. Cheers! James Eagan ---------------------------------------------------------------------- Comment By: John Nagle (nagle) Date: 2007-04-25 16:44 Message: Logged In: YES user_id=5571 Originator: NO Note: If you need this, M2Crypto, a third party replacement for the SSL module, provides the necessary functionality. M2Crypto tends to be a headache to build (it uses SWIG, has version dependencies on SWIG, OpenSSL, the compiler, and CPython, and needs some code from its source repository that isn't in the current release), but the correct functionality is in there. That's what I'm using now. ---------------------------------------------------------------------- Comment By: vila-sf (vila-sf) Date: 2007-04-25 08:35 Message: Logged In: YES user_id=1584517 Originator: NO Like nagle, I have a need for this. But the updated patch is now returning a 404 :-( Since we are at still three interested by that patch, what can we do ? ---------------------------------------------------------------------- Comment By: vila-sf (vila-sf) Date: 2007-04-25 08:33 Message: Logged In: YES user_id=1584517 Originator: NO Like nagle, I have a need for this. But the updated patch is now returning a 404 :-( Since we are at still three interested by that patch, what can we do ? ---------------------------------------------------------------------- Comment By: James Eagan (noonian) Date: 2006-11-30 21:09 Message: Logged In: YES user_id=31389 Originator: YES I'd be happy to make the changes l?wis suggested, but it will be quite a while before I can find the necessary time. If anyone else can update the docs and tests, please let me know! ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-30 20:50 Message: Logged In: YES user_id=21627 Originator: NO The patch is not integrated because nobody had the time to review it; this, in turn, did not happen because we lack reviewers. A quick review reveals that the patch is incomplete: it does not provide changes to the documentation (which it needs to, because it introduces a new feature). The patch also includes no changes to the test suite. ---------------------------------------------------------------------- Comment By: John Nagle (nagle) Date: 2006-11-30 20:13 Message: Logged In: YES user_id=5571 Originator: NO This has been pending for a few months, and there's a fix, but it's not in yet. What's going on? I just had Python accept a totally bogus certificate from "www.amaison.co.uk". The certificate contents are C = -- ST = SomeState L = SomeCity O = SomeOrganization OU = SomeOrganizationalUnit CN = localhost.localdomain emailAddress = root at localhost.localdomain Issuer identity: C = -- ST = SomeState L = SomeCity O = SomeOrganization OU = SomeOrganizationalUnit CN = localhost.localdomain emailAddress = root at localhost.localdomain Python is perfectly happy with that. Which is embarassing. ---------------------------------------------------------------------- Comment By: Gustavo J. A. M. Carneiro (gustavo) Date: 2006-11-09 15:20 Message: Logged In: YES user_id=908 > This patch adds 3 module-level variables to the socket module, which get exposed iff ssl is built in. These variables (ssl_ca_file, ssl_ca_path, and ssl_verify_level) provide programmer-level access to the certificate authorities database and to control what level of certificate verification is performed (by default, none, as is the current behavior). Are you sure it's a good idea to have this kind of 'global' control over certification authorities? Global configurations are handy at first, but they come back and bite us when we least expect it... ---------------------------------------------------------------------- Comment By: James Eagan (noonian) Date: 2006-11-09 14:43 Message: Logged In: YES user_id=31389 Nagle: I haven't heard anything from anyone besides you and jbowes abou this patch here or on the python-dev list, and I haven't had time to follow up. You might have more success via the email list. (Or, if any of the python maintainers is reading this, do you have any suggestions to make this patch more attractive?) ---------------------------------------------------------------------- Comment By: John Nagle (nagle) Date: 2006-11-09 04:14 Message: Logged In: YES user_id=5571 What's the status of this? Is it going in? I have a need for it. Thanks. ---------------------------------------------------------------------- Comment By: James Bowes (jbowes) Date: 2006-06-21 19:43 Message: Logged In: YES user_id=1543815 I put together an updated version of this patch against svn trunk as of June 21, 2006. I also added some additional documentation to the .tex file. Maybe someone with sufficient privilidges (or James, if you're still out there) could attach the updated patch here? the updated patch is at: http://www.dangerouslyinc.com/~bowes/ssl_ca.diff Regards, James Bowes ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1114345&group_id=5470 From noreply at sourceforge.net Wed Apr 25 18:51:34 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 09:51:34 -0700 Subject: [Patches] [ python-Patches-1706989 ] Implementation of @abstractmethod for PEP 3119 Message-ID: Patches item #1706989, was opened at 2007-04-24 18:45 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Implementation of @abstractmethod for PEP 3119 Initial Comment: This implements a new builtin, abstractmethod, which when used as a method decorator declares the method to be abstract, causing the class to be abstract (i.e. it cannot be instantiated). A subclass of an abstract class is still abstract unless it overrides all abstract base methods. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-25 12:51 Message: Logged In: YES user_id=6380 Originator: YES Neil: The intention is that only methods can be abstract. I don't think we should attempt to only check the __isabstractmethod__ attribute for objects that we know are methods; that would be an expensive check to make (you'd have to call __get__ if it exists) and since this is a __magic__ attribute, you void your warrantee if you mess with it. :-) In this version (v3), I've fixed the C nits and done the refactoring you suggested, and also added an optimization: check_abstracts() returns immediately if the base class doesn't have the ABSTRACT flag set. File Added: abstract.diff ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-25 01:35 Message: Logged In: YES user_id=33168 Originator: NO Perhaps this is a better question for the PEP rather than the impl, but can attributes be abstract? class Foo: abstract_override_me = ??? If so, then __isabstractmethod__ might be better named as: __isabstract__. I think this might work: class Abstract: __isabstractmethod__ = True class Foo: abstract_override_me = Abstract() Do you want arbitrary objects to be able to declare their abstractness or should the impl also check that an attribute is callable? check_new_abstracts() should return a Py_ssize_t since it returns the size of a container (set). The return value is already captured in a Py_ssize_t, so it's just the signature (and prototype) that should change. PySet_Add()s return value isn't checked in check_new_abstracts(). It might be nice to factor out the common code between the two new functions into a static helper function. That would get rid of the PySet_Add problem. By calling: PyObject_GetAttrString(meth, "__isabstractmethod__"), that means a new string object is allocated and then thrown away with each call. This could be improved by creating an interned string for "__isabstractmethod__". (I realize this is only when types are created which shouldn't be too often.) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-24 19:31 Message: Logged In: YES user_id=6380 Originator: YES Here's a version that compiles with C89 (GCC 2.96) and doesn't leak the 'fast' object. File Added: abstract.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706989&group_id=5470 From noreply at sourceforge.net Thu Apr 26 01:13:38 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 16:13:38 -0700 Subject: [Patches] [ python-Patches-1707753 ] get status output fix for Win32 Message-ID: Patches item #1707753, was opened at 2007-04-25 18:13 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1707753&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ken Phillis Jr. (dandel1984) Assigned to: Nobody/Anonymous (nobody) Summary: get status output fix for Win32 Initial Comment: the two command output retrieval functions for python are broken on win32, and this patch fixes this, although it is possible that this fix will also work on dos, and os/2, but i am not sure about this... I've already tested it against python 2.5, 2.4 and 2.3, on windows xp and server 2k3. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1707753&group_id=5470 From noreply at sourceforge.net Thu Apr 26 06:26:21 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 25 Apr 2007 21:26:21 -0700 Subject: [Patches] [ python-Patches-1706323 ] Updated ASTVisitor Classes Message-ID: Patches item #1706323, was opened at 2007-04-24 07:30 Message generated for change (Comment added) made by loupgaroublond You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Yaakov Nemoy (loupgaroublond) Assigned to: Nobody/Anonymous (nobody) Summary: Updated ASTVisitor Classes Initial Comment: Ok, So this is really more of a complete rewrite than a bunch of patches, but it's just a picture of what IMHO is a saner way to do this module. I suppose I could refactor alot of it to look more like the old module, but I'm still not sure what would be better, in terms of having seperate visitors to walkers. Either way, the terminology in the original wasn't much clearer. At bare minimum though, i got postorder implemented, so this code has got to be worth something valuable... I hope.... Feedback please. ---------------------------------------------------------------------- >Comment By: Yaakov Nemoy (loupgaroublond) Date: 2007-04-26 06:26 Message: Logged In: YES user_id=1487908 Originator: YES Ok, I made your recommended changes, and added a scanty bit of documentation. Just wondering, how long till such a change would be put upstream? I'm going to use this in a project. File Added: visitor.py ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 23:16 Message: Logged In: YES user_id=764593 Originator: NO > I'll take a look at #3, but I'm not quite familiar with the Py3k changes Don't worry too much -- for the most part, Py3k is just a matter of cleanup. For reasonable code, there usually isn't much difference between class Old: ... and class New(object): ... But this means that when there is a difference, it is a pain to debug. So just stick with the version that will continue to be available (inheriting from object), and backwards-compatibility will take care of itself. ---------------------------------------------------------------------- Comment By: Yaakov Nemoy (loupgaroublond) Date: 2007-04-24 23:03 Message: Logged In: YES user_id=1487908 Originator: YES That looks like a good idea. I just hammered the code out last night, and I'll try to include your changes tonight if I get a chance. I'll take a look at #3, but I'm not quite familiar with the Py3k changes as I should be. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 15:53 Message: Logged In: YES user_id=764593 Originator: NO I haven't looked deeply yet, but I know some feedback quickly is often better than a long wait. I do have a few style suggestions. (1) Tests for None should generally use "is" rather than "==". (2) It helps to understand the code if it is more parallel. Seeing the meth is/is not None cases reversed on the two orders confused me about how similar they were. (3) Inheriting from object (or setting __metaclass__ = type) is probably a good habit, since old-style classes will disappear in Py3K. Normally, it doesn't matter, but debugging the difference is a pain. To make the first two concrete, here is my rewording of ASTVisitor.dispatch: From: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if self._order == self.PREORDER: if meth != None and meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) elif meth == None: self._iterate_over_others(node, *args) elif self._order == self.POSTORDER: if meth == None: self._iterate_over_others(node, *args) if meth != None: self._iterate_over_defaults(node, *args) meth(node, *args) To: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if meth is None: self._iterate_over_others(node, *args) else: if self._order == self.POSTORDER: self._iterate_over_defaults(node, *args) meth(node, *args) elif meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) else: pass # pre-order, and the method said to stop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 From noreply at sourceforge.net Thu Apr 26 10:06:00 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 01:06:00 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 20:48 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None >Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 10:06 Message: Logged In: YES user_id=21627 Originator: NO AFAICT, MOVEFILE_COPY_ALLOWED is not needed to fix 1693753. If the objective of this patch is to establish consistency across platforms, then this should *not* be added, since os.rename won't rename across volumes on Unix, either. So I'm also -1 on the patch in this form. jorend: would you still object if it only did MOVEFILE_REPLACE_EXISTING? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 22:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 22:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Thu Apr 26 10:44:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 01:44:12 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 09:39 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 10:44 Message: Logged In: YES user_id=21627 Originator: NO I guess we'll need a number of iterations. Some observations: - I read the nodeValue specification so that it raises DOMException if you try to set it, and it is "readonly", where I interpret "readonly" as "always null" (because it appears to be the only case where the value is never changing). So __setattr__ should raise that exception, IMO. - I think that the checks for correct names should be moved out of the minidom module, into a more global place (because it might be useful independent of minidom), e.g. xml.names. See also xml.utils.characters in PyXML. - Setting the xmlVersion to 1.1 should raise NOT_SUPPORTED_ERR IMO: we cannot guarantee that we will write out correct XML 1.1 (i.e. a document that is in Unicode normal form NFC). More later. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-25 14:02 Message: Logged In: YES user_id=18139 Originator: YES File Added: minidom-level-1-compliance.3.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-25 08:26 Message: Logged In: YES user_id=33168 Originator: NO Jason, don't worry about the size, I was mostly complaining about the whitespace diffs. I know Tim hadn't done a normalization step for a while. I just did it and am running the tests now. Damn there are a lot! :-( The good news is that your patch will go down. The bad news for you is that you are going to have to regenerate it and there will probably be conflicts. I'm hoping Martin will review it as he's quite knowledgeable about XML. I doubt there is any more that I can contribute to this patch other than to say it needs some expert eyes on it, but generally looks good. The tests should be done within 15 minutes or so. Then I'll check in whitespace normalization. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-25 05:55 Message: Logged In: YES user_id=18139 Originator: YES OK, the updated patch has the suggested changes (Doc for EntityReference; eliminated "pass" and added comment in Node.__setattr__). File Added: minidom-level-1-compliance.2.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 13:53 Message: Logged In: YES user_id=18139 Originator: YES Neal, thanks for looking at this. Size: Yeah. :-\ I don't know what to do about this. There were a lot of bugs, so there are a lot of diffs. :( Whitespace: I checked, and the extra whitespace is definitely in trunk's Lib/test/test_minidom.py; a recent change added it. This patch removes it. (I could revert the whitespace anyway, just to reduce the size of the patch. Let me know if I should do this.) Doc: Oops! I will fix this today and upload a new patch. pass: Thanks, I'll change as suggested. (Just to explain myself-- I used "if name == 'nodeValue': pass; else: ..." because 'nodeValue' is the special case, and the special behavior is "do nothing". But I'm not attached to that wording. I'll change it.) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 08:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 15:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Thu Apr 26 15:06:13 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 06:06:13 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 03:39 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:06 Message: Logged In: YES user_id=18139 Originator: YES Martin, thank you very, very much for looking at this. Happy to iterate. I'll turn it around as fast as I can, probably tomorrow. - Node.nodeValue isn't readonly; setting it raises an exception only if the *node* is readonly. DOM Level 2 clarifies: "When it is defined to be null, setting it has no effect." http://www.w3.org/TR/REC-DOM-Level-1/level-one-core.html#ID-1950641247 http://www.w3.org/TR/DOM-Level-2-Core/core.html#ID-F68D080 - I agree about moving the name-checking code out of minidom. "xml.names" sounds fine. Should it be official and documented? xml.names.is_name(name, [xmlVersion="1.0"]) -> bool Returns True if name matches the Name construction in the specified version of XML. Raises ValueError if xmlVersion is not a known version of XML (currently either "1.0" or "1.1"). - Regarding xmlVersion = "1.1": I don't think writing "fully normalized" XML is required. It's only a "SHOULD" in XML 1.1: "All XML parsed entities (including document entities) SHOULD be fully normalized [...] However, a document is still well-formed even if it is not fully normalized. XML processors SHOULD provide a user option to verify that the document being processed is in fully normalized form, and report to the application whether it is or not." http://www.w3.org/TR/xml11/#sec-normalization-checking DOM Level 3 provides a "normalize-characters" option. Implementations are not required to support it: http://www.w3.org/TR/2004/REC-DOM-Level-3-LS-20040407/load-save.html#parameter-normalize-characters I want to add normalization to writexml() and toxml(), but I hope it can wait until after this patch lands. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 04:44 Message: Logged In: YES user_id=21627 Originator: NO I guess we'll need a number of iterations. Some observations: - I read the nodeValue specification so that it raises DOMException if you try to set it, and it is "readonly", where I interpret "readonly" as "always null" (because it appears to be the only case where the value is never changing). So __setattr__ should raise that exception, IMO. - I think that the checks for correct names should be moved out of the minidom module, into a more global place (because it might be useful independent of minidom), e.g. xml.names. See also xml.utils.characters in PyXML. - Setting the xmlVersion to 1.1 should raise NOT_SUPPORTED_ERR IMO: we cannot guarantee that we will write out correct XML 1.1 (i.e. a document that is in Unicode normal form NFC). More later. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-25 08:02 Message: Logged In: YES user_id=18139 Originator: YES File Added: minidom-level-1-compliance.3.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-25 02:26 Message: Logged In: YES user_id=33168 Originator: NO Jason, don't worry about the size, I was mostly complaining about the whitespace diffs. I know Tim hadn't done a normalization step for a while. I just did it and am running the tests now. Damn there are a lot! :-( The good news is that your patch will go down. The bad news for you is that you are going to have to regenerate it and there will probably be conflicts. I'm hoping Martin will review it as he's quite knowledgeable about XML. I doubt there is any more that I can contribute to this patch other than to say it needs some expert eyes on it, but generally looks good. The tests should be done within 15 minutes or so. Then I'll check in whitespace normalization. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 23:55 Message: Logged In: YES user_id=18139 Originator: YES OK, the updated patch has the suggested changes (Doc for EntityReference; eliminated "pass" and added comment in Node.__setattr__). File Added: minidom-level-1-compliance.2.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 07:53 Message: Logged In: YES user_id=18139 Originator: YES Neal, thanks for looking at this. Size: Yeah. :-\ I don't know what to do about this. There were a lot of bugs, so there are a lot of diffs. :( Whitespace: I checked, and the extra whitespace is definitely in trunk's Lib/test/test_minidom.py; a recent change added it. This patch removes it. (I could revert the whitespace anyway, just to reduce the size of the patch. Let me know if I should do this.) Doc: Oops! I will fix this today and upload a new patch. pass: Thanks, I'll change as suggested. (Just to explain myself-- I used "if name == 'nodeValue': pass; else: ..." because 'nodeValue' is the special case, and the special behavior is "do nothing". But I'm not attached to that wording. I'll change it.) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 02:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 09:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Thu Apr 26 15:25:24 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 06:25:24 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:25 Message: Logged In: YES user_id=18139 Originator: NO loewis: I wish I could say no. I would prefer the new behavior. But the existing behavior is documented and has been there for a long time. People are surely relying on it in some Windows-only program somewhere. And man, when their code breaks, it'll break by unrecoverably clobbering files. :( Still -1, sorry. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 04:06 Message: Logged In: YES user_id=21627 Originator: NO AFAICT, MOVEFILE_COPY_ALLOWED is not needed to fix 1693753. If the objective of this patch is to establish consistency across platforms, then this should *not* be added, since os.rename won't rename across volumes on Unix, either. So I'm also -1 on the patch in this form. jorend: would you still object if it only did MOVEFILE_REPLACE_EXISTING? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 16:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Thu Apr 26 16:27:04 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 07:27:04 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- >Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 10:27 Message: Logged In: YES user_id=984087 Originator: YES I really wish one can reply to comments like in email, by quoting part of original mail. I don't know if the new tracker allows such option. Coming to the point, 1) MOVEFILE_COPY_ALLOWED is not needed to fix 1693753 but is needed if old functionality needs to be maintained. The API MoveFile() does move files across file systems (as per msdn doc). 2) As per os.rename doc, renaming across file systems *may* fail on some unix flavors. That gave me the impression that it *may* succeed on some others. If we can confirm that it *always* fails on *all* flavors, we can update the doc and I can do away with MOVEFILE_COPY_ALLOWED. 3) The patch breaks exiting code behaviour as jorend noted but not patching for that reason alone does not seem right to me, particularly since the patch adds desirable functionality. I don't think coming up with a new function name is a good idea. Is it ok to change the behaviour of rename() based on some setting in "os" module? If nothing else, the patch can be applied to 3.0 and 2.x users can be given warnings. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:25 Message: Logged In: YES user_id=18139 Originator: NO loewis: I wish I could say no. I would prefer the new behavior. But the existing behavior is documented and has been there for a long time. People are surely relying on it in some Windows-only program somewhere. And man, when their code breaks, it'll break by unrecoverably clobbering files. :( Still -1, sorry. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 04:06 Message: Logged In: YES user_id=21627 Originator: NO AFAICT, MOVEFILE_COPY_ALLOWED is not needed to fix 1693753. If the objective of this patch is to establish consistency across platforms, then this should *not* be added, since os.rename won't rename across volumes on Unix, either. So I'm also -1 on the patch in this form. jorend: would you still object if it only did MOVEFILE_REPLACE_EXISTING? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 16:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Thu Apr 26 16:27:14 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 07:27:14 -0700 Subject: [Patches] [ python-Patches-1707753 ] get status output fix for Win32 Message-ID: Patches item #1707753, was opened at 2007-04-25 19:13 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1707753&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ken Phillis Jr. (dandel1984) Assigned to: Nobody/Anonymous (nobody) Summary: get status output fix for Win32 Initial Comment: the two command output retrieval functions for python are broken on win32, and this patch fixes this, although it is possible that this fix will also work on dos, and os/2, but i am not sure about this... I've already tested it against python 2.5, 2.4 and 2.3, on windows xp and server 2k3. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 10:27 Message: Logged In: YES user_id=18139 Originator: NO Ken, thanks for the patch! To be considered for Python 2.6, this patch would have to update documentation (Doc/lib) and tests (Lib/test) as well. But more importantly: This entire module is likely to be deprecated in Python 2.6, in favor of the subprocess module (which will gain new functions, subprocess.get_output() and subprocess.get_status_output().) There was a long discussion about it on python-dev: (start of a long thread) http://mail.python.org/pipermail/python-dev/2007-March/071929.html (even more) http://mail.python.org/pipermail/python-dev/2007-March/072046.html If that happens, there's no particular reason to do this, right? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1707753&group_id=5470 From noreply at sourceforge.net Thu Apr 26 17:11:06 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 08:11:06 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 11:11 Message: Logged In: YES user_id=18139 Originator: NO draghuram, Thanks, now I understand about MOVEFILE_COPY_ALLOWED. I don't think 3.0 is the answer--it's not the "subtle breakage" release. Eventually someone will do for the filesystem what the subprocess module did for processes: implement a *real* compatibility layer that behaves the same across all supported platforms. Until that sunny day, I think we have to live with the quirks. :-\ ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 10:27 Message: Logged In: YES user_id=984087 Originator: YES I really wish one can reply to comments like in email, by quoting part of original mail. I don't know if the new tracker allows such option. Coming to the point, 1) MOVEFILE_COPY_ALLOWED is not needed to fix 1693753 but is needed if old functionality needs to be maintained. The API MoveFile() does move files across file systems (as per msdn doc). 2) As per os.rename doc, renaming across file systems *may* fail on some unix flavors. That gave me the impression that it *may* succeed on some others. If we can confirm that it *always* fails on *all* flavors, we can update the doc and I can do away with MOVEFILE_COPY_ALLOWED. 3) The patch breaks exiting code behaviour as jorend noted but not patching for that reason alone does not seem right to me, particularly since the patch adds desirable functionality. I don't think coming up with a new function name is a good idea. Is it ok to change the behaviour of rename() based on some setting in "os" module? If nothing else, the patch can be applied to 3.0 and 2.x users can be given warnings. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:25 Message: Logged In: YES user_id=18139 Originator: NO loewis: I wish I could say no. I would prefer the new behavior. But the existing behavior is documented and has been there for a long time. People are surely relying on it in some Windows-only program somewhere. And man, when their code breaks, it'll break by unrecoverably clobbering files. :( Still -1, sorry. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 04:06 Message: Logged In: YES user_id=21627 Originator: NO AFAICT, MOVEFILE_COPY_ALLOWED is not needed to fix 1693753. If the objective of this patch is to establish consistency across platforms, then this should *not* be added, since os.rename won't rename across volumes on Unix, either. So I'm also -1 on the patch in this form. jorend: would you still object if it only did MOVEFILE_REPLACE_EXISTING? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 16:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Thu Apr 26 20:43:25 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 11:43:25 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- >Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 14:43 Message: Logged In: YES user_id=984087 Originator: YES FWIW, I googled "python os.rename windows" only to find a (very) old discussion on python-dev about exact same topic. http://mail.python.org/pipermail/python-dev/2001-May/014972.html The idea of having optional parameter to os.rename() is interesting. What do you think? ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 11:11 Message: Logged In: YES user_id=18139 Originator: NO draghuram, Thanks, now I understand about MOVEFILE_COPY_ALLOWED. I don't think 3.0 is the answer--it's not the "subtle breakage" release. Eventually someone will do for the filesystem what the subprocess module did for processes: implement a *real* compatibility layer that behaves the same across all supported platforms. Until that sunny day, I think we have to live with the quirks. :-\ ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 10:27 Message: Logged In: YES user_id=984087 Originator: YES I really wish one can reply to comments like in email, by quoting part of original mail. I don't know if the new tracker allows such option. Coming to the point, 1) MOVEFILE_COPY_ALLOWED is not needed to fix 1693753 but is needed if old functionality needs to be maintained. The API MoveFile() does move files across file systems (as per msdn doc). 2) As per os.rename doc, renaming across file systems *may* fail on some unix flavors. That gave me the impression that it *may* succeed on some others. If we can confirm that it *always* fails on *all* flavors, we can update the doc and I can do away with MOVEFILE_COPY_ALLOWED. 3) The patch breaks exiting code behaviour as jorend noted but not patching for that reason alone does not seem right to me, particularly since the patch adds desirable functionality. I don't think coming up with a new function name is a good idea. Is it ok to change the behaviour of rename() based on some setting in "os" module? If nothing else, the patch can be applied to 3.0 and 2.x users can be given warnings. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:25 Message: Logged In: YES user_id=18139 Originator: NO loewis: I wish I could say no. I would prefer the new behavior. But the existing behavior is documented and has been there for a long time. People are surely relying on it in some Windows-only program somewhere. And man, when their code breaks, it'll break by unrecoverably clobbering files. :( Still -1, sorry. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 04:06 Message: Logged In: YES user_id=21627 Originator: NO AFAICT, MOVEFILE_COPY_ALLOWED is not needed to fix 1693753. If the objective of this patch is to establish consistency across platforms, then this should *not* be added, since os.rename won't rename across volumes on Unix, either. So I'm also -1 on the patch in this form. jorend: would you still object if it only did MOVEFILE_REPLACE_EXISTING? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 16:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Thu Apr 26 20:47:48 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 11:47:48 -0700 Subject: [Patches] [ python-Patches-1706323 ] Updated ASTVisitor Classes Message-ID: Patches item #1706323, was opened at 2007-04-24 01:30 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Yaakov Nemoy (loupgaroublond) Assigned to: Nobody/Anonymous (nobody) Summary: Updated ASTVisitor Classes Initial Comment: Ok, So this is really more of a complete rewrite than a bunch of patches, but it's just a picture of what IMHO is a saner way to do this module. I suppose I could refactor alot of it to look more like the old module, but I'm still not sure what would be better, in terms of having seperate visitors to walkers. Either way, the terminology in the original wasn't much clearer. At bare minimum though, i got postorder implemented, so this code has got to be worth something valuable... I hope.... Feedback please. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-26 14:47 Message: Logged In: YES user_id=764593 Originator: NO You are welcome to use it yourself, and ship it with your project. It almost certainly won't be added to the standard distribution until the next feature release. PEP 361 http://www.python.org/dev/peps/pep-0361/ says that is targeted around April of 2008. (Alphas will be available sooner, particularly for python 3.x) ---------------------------------------------------------------------- Comment By: Yaakov Nemoy (loupgaroublond) Date: 2007-04-26 00:26 Message: Logged In: YES user_id=1487908 Originator: YES Ok, I made your recommended changes, and added a scanty bit of documentation. Just wondering, how long till such a change would be put upstream? I'm going to use this in a project. File Added: visitor.py ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 17:16 Message: Logged In: YES user_id=764593 Originator: NO > I'll take a look at #3, but I'm not quite familiar with the Py3k changes Don't worry too much -- for the most part, Py3k is just a matter of cleanup. For reasonable code, there usually isn't much difference between class Old: ... and class New(object): ... But this means that when there is a difference, it is a pain to debug. So just stick with the version that will continue to be available (inheriting from object), and backwards-compatibility will take care of itself. ---------------------------------------------------------------------- Comment By: Yaakov Nemoy (loupgaroublond) Date: 2007-04-24 17:03 Message: Logged In: YES user_id=1487908 Originator: YES That looks like a good idea. I just hammered the code out last night, and I'll try to include your changes tonight if I get a chance. I'll take a look at #3, but I'm not quite familiar with the Py3k changes as I should be. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2007-04-24 09:53 Message: Logged In: YES user_id=764593 Originator: NO I haven't looked deeply yet, but I know some feedback quickly is often better than a long wait. I do have a few style suggestions. (1) Tests for None should generally use "is" rather than "==". (2) It helps to understand the code if it is more parallel. Seeing the meth is/is not None cases reversed on the two orders confused me about how similar they were. (3) Inheriting from object (or setting __metaclass__ = type) is probably a good habit, since old-style classes will disappear in Py3K. Normally, it doesn't matter, but debugging the difference is a pain. To make the first two concrete, here is my rewording of ASTVisitor.dispatch: From: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if self._order == self.PREORDER: if meth != None and meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) elif meth == None: self._iterate_over_others(node, *args) elif self._order == self.POSTORDER: if meth == None: self._iterate_over_others(node, *args) if meth != None: self._iterate_over_defaults(node, *args) meth(node, *args) To: def dispatch(self, node, *args): klass = node.__class__ meth = self._get_method_for_className(klass) if meth is None: self._iterate_over_others(node, *args) else: if self._order == self.POSTORDER: self._iterate_over_defaults(node, *args) meth(node, *args) elif meth(node, *args) != self.NO_CONTINUE: self._iterate_over_defaults(node, *args) else: pass # pre-order, and the method said to stop ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706323&group_id=5470 From noreply at sourceforge.net Thu Apr 26 22:16:58 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 13:16:58 -0700 Subject: [Patches] [ python-Patches-1708353 ] Make isinstance/issubclass overloadable Message-ID: Patches item #1708353, was opened at 2007-04-26 16:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1708353&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Make isinstance/issubclass overloadable Initial Comment: I came up with a fairly simple way to overload isinstance() and issubclass(). The class that is the second argument can define a (class) method named __instancecheck__ or __subclasscheck__ which, if present, will be called *instead* of the normal approach. The names are different to remind users that the calling convention is the opposite -- isinstance(x, C) maps to C.__instancecheck__(x). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1708353&group_id=5470 From noreply at sourceforge.net Thu Apr 26 22:34:23 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 13:34:23 -0700 Subject: [Patches] [ python-Patches-1708353 ] Make isinstance/issubclass overloadable Message-ID: Patches item #1708353, was opened at 2007-04-26 16:16 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1708353&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Make isinstance/issubclass overloadable Initial Comment: I came up with a fairly simple way to overload isinstance() and issubclass(). The class that is the second argument can define a (class) method named __instancecheck__ or __subclasscheck__ which, if present, will be called *instead* of the normal approach. The names are different to remind users that the calling convention is the opposite -- isinstance(x, C) maps to C.__instancecheck__(x). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-26 16:34 Message: Logged In: YES user_id=6380 Originator: YES Oops, forget the first attempt; isinstance(Integer(), Integer) would be False! Now it's True, and more tests are added. File Added: typechecks.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1708353&group_id=5470 From noreply at sourceforge.net Thu Apr 26 23:27:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 14:27:12 -0700 Subject: [Patches] [ python-Patches-1708353 ] Make isinstance/issubclass overloadable for PEP 3119 Message-ID: Patches item #1708353, was opened at 2007-04-26 16:16 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1708353&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) >Summary: Make isinstance/issubclass overloadable for PEP 3119 Initial Comment: I came up with a fairly simple way to overload isinstance() and issubclass(). The class that is the second argument can define a (class) method named __instancecheck__ or __subclasscheck__ which, if present, will be called *instead* of the normal approach. The names are different to remind users that the calling convention is the opposite -- isinstance(x, C) maps to C.__instancecheck__(x). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-26 17:27 Message: Logged In: YES user_id=6380 Originator: YES I'll add a description and motivation for this to PEP 3119. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2007-04-26 16:34 Message: Logged In: YES user_id=6380 Originator: YES Oops, forget the first attempt; isinstance(Integer(), Integer) would be False! Now it's True, and more tests are added. File Added: typechecks.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1708353&group_id=5470 From noreply at sourceforge.net Thu Apr 26 23:52:08 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 14:52:08 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 20:48 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 23:52 Message: Logged In: YES user_id=21627 Originator: NO draghuram, I think this should be discussed on python-dev. I'd be in favor of implementing subtle breakage, but I have been criticized for that view, also. I agree with jorend that if there is reason to reject it in 2.6, there is also reason to reject it forever. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 20:43 Message: Logged In: YES user_id=984087 Originator: YES FWIW, I googled "python os.rename windows" only to find a (very) old discussion on python-dev about exact same topic. http://mail.python.org/pipermail/python-dev/2001-May/014972.html The idea of having optional parameter to os.rename() is interesting. What do you think? ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 17:11 Message: Logged In: YES user_id=18139 Originator: NO draghuram, Thanks, now I understand about MOVEFILE_COPY_ALLOWED. I don't think 3.0 is the answer--it's not the "subtle breakage" release. Eventually someone will do for the filesystem what the subprocess module did for processes: implement a *real* compatibility layer that behaves the same across all supported platforms. Until that sunny day, I think we have to live with the quirks. :-\ ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 16:27 Message: Logged In: YES user_id=984087 Originator: YES I really wish one can reply to comments like in email, by quoting part of original mail. I don't know if the new tracker allows such option. Coming to the point, 1) MOVEFILE_COPY_ALLOWED is not needed to fix 1693753 but is needed if old functionality needs to be maintained. The API MoveFile() does move files across file systems (as per msdn doc). 2) As per os.rename doc, renaming across file systems *may* fail on some unix flavors. That gave me the impression that it *may* succeed on some others. If we can confirm that it *always* fails on *all* flavors, we can update the doc and I can do away with MOVEFILE_COPY_ALLOWED. 3) The patch breaks exiting code behaviour as jorend noted but not patching for that reason alone does not seem right to me, particularly since the patch adds desirable functionality. I don't think coming up with a new function name is a good idea. Is it ok to change the behaviour of rename() based on some setting in "os" module? If nothing else, the patch can be applied to 3.0 and 2.x users can be given warnings. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 15:25 Message: Logged In: YES user_id=18139 Originator: NO loewis: I wish I could say no. I would prefer the new behavior. But the existing behavior is documented and has been there for a long time. People are surely relying on it in some Windows-only program somewhere. And man, when their code breaks, it'll break by unrecoverably clobbering files. :( Still -1, sorry. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 10:06 Message: Logged In: YES user_id=21627 Originator: NO AFAICT, MOVEFILE_COPY_ALLOWED is not needed to fix 1693753. If the objective of this patch is to establish consistency across platforms, then this should *not* be added, since os.rename won't rename across volumes on Unix, either. So I'm also -1 on the patch in this form. jorend: would you still object if it only did MOVEFILE_REPLACE_EXISTING? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 22:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 22:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Fri Apr 27 03:14:45 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 18:14:45 -0700 Subject: [Patches] [ python-Patches-1707753 ] get status output fix for Win32 Message-ID: Patches item #1707753, was opened at 2007-04-25 18:13 Message generated for change (Comment added) made by dandel1984 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1707753&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ken Phillis Jr. (dandel1984) Assigned to: Nobody/Anonymous (nobody) Summary: get status output fix for Win32 Initial Comment: the two command output retrieval functions for python are broken on win32, and this patch fixes this, although it is possible that this fix will also work on dos, and os/2, but i am not sure about this... I've already tested it against python 2.5, 2.4 and 2.3, on windows xp and server 2k3. ---------------------------------------------------------------------- >Comment By: Ken Phillis Jr. (dandel1984) Date: 2007-04-26 20:14 Message: Logged In: YES user_id=956063 Originator: YES there is still a reason to do this, except for it'll haft to be integrated into the new module, because of how windows/dos handle the command line input, and most people will try working it as is, which is just simply put in a windows/dos command and expect it to work like it does in unix based systems, at which this does the job really well... a sample of that would be to call it like so: commands.get_output("echo Testing") as for tests I'll get that generated fast, so expect something to be posted in a few days. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:27 Message: Logged In: YES user_id=18139 Originator: NO Ken, thanks for the patch! To be considered for Python 2.6, this patch would have to update documentation (Doc/lib) and tests (Lib/test) as well. But more importantly: This entire module is likely to be deprecated in Python 2.6, in favor of the subprocess module (which will gain new functions, subprocess.get_output() and subprocess.get_status_output().) There was a long discussion about it on python-dev: (start of a long thread) http://mail.python.org/pipermail/python-dev/2007-March/071929.html (even more) http://mail.python.org/pipermail/python-dev/2007-March/072046.html If that happens, there's no particular reason to do this, right? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1707753&group_id=5470 From noreply at sourceforge.net Fri Apr 27 06:55:14 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 26 Apr 2007 21:55:14 -0700 Subject: [Patches] [ python-Patches-1704547 ] Use MoveFileEx() to implement os.rename() on windows Message-ID: Patches item #1704547, was opened at 2007-04-20 14:48 Message generated for change (Comment added) made by draghuram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Raghuram Devarakonda (draghuram) Assigned to: Martin v. L?wis (loewis) Summary: Use MoveFileEx() to implement os.rename() on windows Initial Comment: This patch is to fix bug 1693753. Basically, os.rename(src, dst) fails on windows if dst already exists. This is because MoveFile() is used to implement rename. This patch replaces this API with MoveFileEx() and uses the flag MOVEFILE_REPLACE_EXISTING causing the API to replace the dst if it already exists. This brings the behaviour in line with unix. Note that, I also use the flag MOVEFILE_COPY_ALLOWED which is required if dst is on a different volume. However, moving to a different volume may not be atomic operation (I am not sure about this) as the msdn doc says that move in this case is simulated with CopyFile() and DeleteFile(). The patch also includes a test case and doc update. Please let me know if the location of test case and name of test function are ok. Also, MoveFileEx() is only available on NT+ windows but it should be ok as support for Win9x, WinME is removed in 2.6. ---------------------------------------------------------------------- >Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-27 00:55 Message: Logged In: YES user_id=984087 Originator: YES Ok. I will post to python-dev about this. BTW, have either of you found any reference to MOVEFILE_REPLACE_EXISTING being atomic operation (for intra-file system moving of course)? I couldn't find any direct reference. Raghu. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 17:52 Message: Logged In: YES user_id=21627 Originator: NO draghuram, I think this should be discussed on python-dev. I'd be in favor of implementing subtle breakage, but I have been criticized for that view, also. I agree with jorend that if there is reason to reject it in 2.6, there is also reason to reject it forever. ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 14:43 Message: Logged In: YES user_id=984087 Originator: YES FWIW, I googled "python os.rename windows" only to find a (very) old discussion on python-dev about exact same topic. http://mail.python.org/pipermail/python-dev/2001-May/014972.html The idea of having optional parameter to os.rename() is interesting. What do you think? ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 11:11 Message: Logged In: YES user_id=18139 Originator: NO draghuram, Thanks, now I understand about MOVEFILE_COPY_ALLOWED. I don't think 3.0 is the answer--it's not the "subtle breakage" release. Eventually someone will do for the filesystem what the subprocess module did for processes: implement a *real* compatibility layer that behaves the same across all supported platforms. Until that sunny day, I think we have to live with the quirks. :-\ ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-26 10:27 Message: Logged In: YES user_id=984087 Originator: YES I really wish one can reply to comments like in email, by quoting part of original mail. I don't know if the new tracker allows such option. Coming to the point, 1) MOVEFILE_COPY_ALLOWED is not needed to fix 1693753 but is needed if old functionality needs to be maintained. The API MoveFile() does move files across file systems (as per msdn doc). 2) As per os.rename doc, renaming across file systems *may* fail on some unix flavors. That gave me the impression that it *may* succeed on some others. If we can confirm that it *always* fails on *all* flavors, we can update the doc and I can do away with MOVEFILE_COPY_ALLOWED. 3) The patch breaks exiting code behaviour as jorend noted but not patching for that reason alone does not seem right to me, particularly since the patch adds desirable functionality. I don't think coming up with a new function name is a good idea. Is it ok to change the behaviour of rename() based on some setting in "os" module? If nothing else, the patch can be applied to 3.0 and 2.x users can be given warnings. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:25 Message: Logged In: YES user_id=18139 Originator: NO loewis: I wish I could say no. I would prefer the new behavior. But the existing behavior is documented and has been there for a long time. People are surely relying on it in some Windows-only program somewhere. And man, when their code breaks, it'll break by unrecoverably clobbering files. :( Still -1, sorry. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 04:06 Message: Logged In: YES user_id=21627 Originator: NO AFAICT, MOVEFILE_COPY_ALLOWED is not needed to fix 1693753. If the objective of this patch is to establish consistency across platforms, then this should *not* be added, since os.rename won't rename across volumes on Unix, either. So I'm also -1 on the patch in this form. jorend: would you still object if it only did MOVEFILE_REPLACE_EXISTING? ---------------------------------------------------------------------- Comment By: Raghuram Devarakonda (draghuram) Date: 2007-04-20 16:56 Message: Logged In: YES user_id=984087 Originator: YES > This changes the documented behavior of a commonly used function. Right. If this change is considered too big for 2.6, may be it can be applied to 3.0? > FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally > isolated. Other processes can see the new file being created, and watch > its size increase, while the old one still exists. It isn't atomic, > either: in certain error cases, e.g. if the process's permission to write > the target file is suddenly revoked, it will fail after making changes to > the filesystem. True. But isn't this the case with MoveFile() too? I couldn't find any clear mention about transactional behaviour of either MoveFile() or MoveFileEx(). Same goes for atomicity. > Also-- it looks like the test leaves one of the temp files lying around! I can take care of that. While I think about it, it is perhaps not correct for this test function to be in Win32ErrorTests. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-20 16:47 Message: Logged In: YES user_id=18139 Originator: NO -1. This changes the documented behavior of a commonly used function. FWIW: MoveFileEx() with MOVEFILE_COPY_ALLOWED isn't transactionally isolated. Other processes can see the new file being created, and watch its size increase, while the old one still exists. It isn't atomic, either: in certain error cases, e.g. if the process's permission to write the target file is suddenly revoked, it will fail after making changes to the filesystem. Also-- it looks like the test leaves one of the temp files lying around! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704547&group_id=5470 From noreply at sourceforge.net Sat Apr 28 04:40:29 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 27 Apr 2007 19:40:29 -0700 Subject: [Patches] [ python-Patches-1704134 ] minidom Level 1 DOM compliance Message-ID: Patches item #1704134, was opened at 2007-04-20 03:39 Message generated for change (Comment added) made by jorend You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jason Orendorff (jorend) Assigned to: Nobody/Anonymous (nobody) Summary: minidom Level 1 DOM compliance Initial Comment: Tested on: Mac OS X 10.4.9 This patch fixes numerous bugs in xml.dom.minidom and expatbuilder. It fixes all the small-to-middling bugs in minidom's DOM Level 1 compliance that I'm aware of; only large bugs remain (see below). Changes: These are mainly fixes for bugs found by the W3C DOM Test Suite for DOM Level 1. Python 2.5 fails over 120 of these tests; I got the number down to 48. - Exposed expat's XML_GetSpecifiedAttributeCount() as a method of pyexpat parser objects. (This is needed to set Attr.specified correctly.) Documented the new method in Doc/lib/libpyexpat.tex. - Attributes that got default values from the DTD didn't show up in the DOM. (This was a violation of the XML 1.0 spec! See http://www.w3.org/TR/REC-xml/#proc-types section 5.1, "Validating and Non-Validating Processors". Even non-validating processors must "supply default attribute values" based on a certain subset of the DTD!) - Attr.specified is now set correctly. Before, it was always False (should have been True). - Inserting a node into one of its descendants caused an infinite loop! :-) Now it throws HierarchyMalarkey, per the spec. - Many error conditions specified in the DOM were not detected. In particular, InvalidCharacterErr was never raised. The new version does a lot more checking. - Assigning to nodeValue is now a no-op for node types where it's defined to be null. - Document.createEntityReference() is implemented. It returns an EntityReference node, but the node is not populated from the DTD. (That is, the new EntityReference implementation is compliant as far as it goes, but incomplete.) - Element.removeAttributeNode(attr) now raises NotFoundErr if attr belongs to some other Element and merely has the same name as an attribute of this Element. - Element.setAttributeNode() would sometimes return None erroneously. - Element.removeAttributeNode() now returns the removed node. - Several CharacterData methods would incorrectly throw if you passed node.length as the index. - Added Document.xmlVersion (from DOM Level 3). This affects INVALID_CHARACTER_ERR checking as specified. - Added tests for all of the above. - Removed trailing whitespace from lines in Lib/test/test_minidom.py. - Deleted obsolete gc testing from test_minidom. - In one or two places, broke very large asserts into many small asserts. (I was debugging something. This change is inessential, but it's a good change, so I kept it.) DOM Level 1 bugs remaining: - A lot of the readonly properties are not implemented as readonly. This would be easy to fix with new-style classes, but these are old-style classes that are using property() for a few things--I haven't tried to understand it yet. I'm putting this off until the present patch lands. - All NodeLists should be live views, even the one returned by getElementsByName(). It will be hard to fix this while retaining pickle backward compatibility, and still harder to do it without hurting performance. - Attribute nodes' nodeValue and childNodes are supposed to stay in sync. This has all the same problems. - EntityReference nodes should be populated with child nodes. The descendants of EntityReference nodes should be readonly. This is slightly less of a headache. I haven't even tried to run DOMTS level2 tests yet. I'm sure it'll be pretty gruesome. ---------------------------------------------------------------------- >Comment By: Jason Orendorff (jorend) Date: 2007-04-27 22:40 Message: Logged In: YES user_id=18139 Originator: YES Martin, This version of the patch moves name-checking into a separate module, "xml.names". But for now it is undocumented (intentionally). Also, DOMImplementation.hasFeature("XMLVersion", v) now returns True for v in ("1.0", "1.1"), and setting doc.xmlVersion to any other value raises NotSupportedErr. (Both these things are specified in DOM Level 3 Core under the Document.xmlVersion attribute.) Added a test for this, too. I didn't make any other changes. Let me know if I'm wrong about those other two things. File Added: minidom-level-1-compliance.4.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-26 09:06 Message: Logged In: YES user_id=18139 Originator: YES Martin, thank you very, very much for looking at this. Happy to iterate. I'll turn it around as fast as I can, probably tomorrow. - Node.nodeValue isn't readonly; setting it raises an exception only if the *node* is readonly. DOM Level 2 clarifies: "When it is defined to be null, setting it has no effect." http://www.w3.org/TR/REC-DOM-Level-1/level-one-core.html#ID-1950641247 http://www.w3.org/TR/DOM-Level-2-Core/core.html#ID-F68D080 - I agree about moving the name-checking code out of minidom. "xml.names" sounds fine. Should it be official and documented? xml.names.is_name(name, [xmlVersion="1.0"]) -> bool Returns True if name matches the Name construction in the specified version of XML. Raises ValueError if xmlVersion is not a known version of XML (currently either "1.0" or "1.1"). - Regarding xmlVersion = "1.1": I don't think writing "fully normalized" XML is required. It's only a "SHOULD" in XML 1.1: "All XML parsed entities (including document entities) SHOULD be fully normalized [...] However, a document is still well-formed even if it is not fully normalized. XML processors SHOULD provide a user option to verify that the document being processed is in fully normalized form, and report to the application whether it is or not." http://www.w3.org/TR/xml11/#sec-normalization-checking DOM Level 3 provides a "normalize-characters" option. Implementations are not required to support it: http://www.w3.org/TR/2004/REC-DOM-Level-3-LS-20040407/load-save.html#parameter-normalize-characters I want to add normalization to writexml() and toxml(), but I hope it can wait until after this patch lands. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-26 04:44 Message: Logged In: YES user_id=21627 Originator: NO I guess we'll need a number of iterations. Some observations: - I read the nodeValue specification so that it raises DOMException if you try to set it, and it is "readonly", where I interpret "readonly" as "always null" (because it appears to be the only case where the value is never changing). So __setattr__ should raise that exception, IMO. - I think that the checks for correct names should be moved out of the minidom module, into a more global place (because it might be useful independent of minidom), e.g. xml.names. See also xml.utils.characters in PyXML. - Setting the xmlVersion to 1.1 should raise NOT_SUPPORTED_ERR IMO: we cannot guarantee that we will write out correct XML 1.1 (i.e. a document that is in Unicode normal form NFC). More later. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-25 08:02 Message: Logged In: YES user_id=18139 Originator: YES File Added: minidom-level-1-compliance.3.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-25 02:26 Message: Logged In: YES user_id=33168 Originator: NO Jason, don't worry about the size, I was mostly complaining about the whitespace diffs. I know Tim hadn't done a normalization step for a while. I just did it and am running the tests now. Damn there are a lot! :-( The good news is that your patch will go down. The bad news for you is that you are going to have to regenerate it and there will probably be conflicts. I'm hoping Martin will review it as he's quite knowledgeable about XML. I doubt there is any more that I can contribute to this patch other than to say it needs some expert eyes on it, but generally looks good. The tests should be done within 15 minutes or so. Then I'll check in whitespace normalization. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 23:55 Message: Logged In: YES user_id=18139 Originator: YES OK, the updated patch has the suggested changes (Doc for EntityReference; eliminated "pass" and added comment in Node.__setattr__). File Added: minidom-level-1-compliance.2.patch ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-24 07:53 Message: Logged In: YES user_id=18139 Originator: YES Neal, thanks for looking at this. Size: Yeah. :-\ I don't know what to do about this. There were a lot of bugs, so there are a lot of diffs. :( Whitespace: I checked, and the extra whitespace is definitely in trunk's Lib/test/test_minidom.py; a recent change added it. This patch removes it. (I could revert the whitespace anyway, just to reduce the size of the patch. Let me know if I should do this.) Doc: Oops! I will fix this today and upload a new patch. pass: Thanks, I'll change as suggested. (Just to explain myself-- I used "if name == 'nodeValue': pass; else: ..." because 'nodeValue' is the special case, and the special behavior is "do nothing". But I'm not attached to that wording. I'll change it.) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2007-04-24 02:58 Message: Logged In: YES user_id=33168 Originator: NO Wow this patch is big! I notice a lot of whitespace changes, can you drop those from the patch, it will make it easier to review. I don't see any extra whitespace in the original, so maybe if you ran Tools/scripts/reindent.py over the code that would clean it up. I think that's what Tim uses to normalize whitespace. The new tests are good to see. I see you added doc for a new method in xmlparser, but the new class EntityReference from minidom.py is not documented. In xml/dom/minidom.py, it seems odd to see if cond: pass else: # do something. What about inverting the condition so pass/else is not necessary. The comment could be improved here as to why this is a no-op. The patch itself looked fine to me. I didn't review in complete context (ie, I didn't apply it and review all the code), just the patch itself. I don't know enough about minidom to know if this change it good or not. Other than what I noted above, the patch looks decent on the surface. ---------------------------------------------------------------------- Comment By: Jason Orendorff (jorend) Date: 2007-04-23 09:40 Message: Logged In: YES user_id=18139 Originator: YES Justification for the most significant changes: 1. I added a __setattr__() method to class xml.dom.minidom.Node. This means existing code that subclasses any of the Node classes *and* overrides __setattr__() *and* isn't calling the base class __setattr__... will not get the new __setattr__ functionality, which basically implements a DOM-compatibility quirk. I think it's OK. :) 2. There's a flag in pyexpat that lets you turn off default attribute values. The flag is marked "use with caution", because turning this off is not XML-compliant. expatbuilder was using the flag. I can't tell why. There was no comment in the code. I think it was just a mistake; I changed it, and minidom passes more DOMTS tests as a result. 3. To implement Attr.specified, I had to expose another Expat API via pyexpat. This was actually a pretty minor change, but I mention it because I'm sure anyone opening this patch will be shocked that it touches any C code. :) 4. Added some big regexes to check for non-XML-compliant names and raise InvalidCharacterErr. I didn't see a better way to do this. If anyone is worried about the performance hit from doing these checks, I'll look at it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1704134&group_id=5470 From noreply at sourceforge.net Sat Apr 28 08:35:31 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 27 Apr 2007 23:35:31 -0700 Subject: [Patches] [ python-Patches-1709112 ] test_1686475 of test_os & pagefile.sys Message-ID: Patches item #1709112, was opened at 2007-04-28 06:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: A.B., Khalid (abkhd) Assigned to: Nobody/Anonymous (nobody) Summary: test_1686475 of test_os & pagefile.sys Initial Comment: test_1686475 of test_os appears to rely on the existence of "pagefile.sys" on the C drive like so: def test_1686475(self): # Verify that an open file can be stat'ed try: os.stat(r"c:\pagefile.sys") except WindowsError, e: if e == 2: # file does not exist; cannot run test return self.fail("Could not stat pagefile.sys") But since pagefile.sys may not be on every C drive and since the exception object e is not a numeric then that test is probably going to fail if the pagefile.sys does not exist where it is expected. In such a case Windows returns "[Error 5] Access is denied: 'c:\\pagefile.sys'". So we need to test if e.winerror == 5 to skip this and return without failing. Patch "test_os.patch" is attached. By the way, I see the same problem in the trunk, so maybe this need to be applied there as well. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 From noreply at sourceforge.net Sat Apr 28 19:03:06 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 28 Apr 2007 10:03:06 -0700 Subject: [Patches] [ python-Patches-1706039 ] Added clearerr() to clear EOF state Message-ID: Patches item #1706039, was opened at 2007-04-23 17:21 Message generated for change (Comment added) made by josm You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: jos (josm) Assigned to: Nobody/Anonymous (nobody) Summary: Added clearerr() to clear EOF state Initial Comment: This patch is to fix bug 1523853. Now file.read() works as written in the doc, which says "An empty string is returned when EOF is encountered immediately." Before this fix, An empty string was returned even when the last read() encountered EOF. ---------------------------------------------------------------------- >Comment By: jos (josm) Date: 2007-04-28 17:03 Message: Logged In: YES user_id=1776568 Originator: YES New patch Attached. I believe this one includes all belopolsky's suggestions. Now all dirty jobs are done in Py_UniversalNewlineFread: 1. calls clearerr if ferror or feof 2. set IOErrorr exception and return The caller just checks whether IOError is occurred or not and does appropriate jobs. The attachment also contains unittest for this problem. File Added: eof.diff ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-25 03:14 Message: Logged In: YES user_id=835142 Originator: NO I have just verified that this issue affects readlines and readinto methods on Mac OS X as well. This tells me that rather than patching all the individual methods, maybe Py_UniversalNewlineFread should be fixed instead. I believe that the current situation where a function that looks like Python API reports errors by setting flags on a FILE object instead of settin the an exception is against python's design. I suggest changing Py_UniversalNewlineFgets so that if fread returns 0, it checks whether that was due to EOF or an error and sets an exception in a case of an error and always calls clearerr. ---------------------------------------------------------------------- Comment By: jos (josm) Date: 2007-04-24 17:15 Message: Logged In: YES user_id=1776568 Originator: YES I confirmed that fread can read data from fp in EOF state on Linux. On OS X, freading from fp in EOF state doesn't work, just return 0. I tested this behavior using the following code. ---------------------------- #include #include main(void) { FILE *fp1, *fp2; int n; char buf[128]; fp1 = fopen("spam.txt", "w+"); fputs("foo", fp1); fseek(fp1, 0L, SEEK_END); if (fread(buf, 1, sizeof(buf), fp1) == 0) fprintf(stderr, "first fread failed\n"); fp2 = fopen("spam.txt", "a+"); fputs("bar", fp2); fclose(fp2); if (feof(fp1)) fprintf(stderr, "after appending some text, fp1 is still eof\n"); if (fread(buf, 1, sizeof(buf), fp1) == 0) fprintf(stderr, "second fread failed\n"); printf("buf: %s\n", buf); fclose(fp1); return 0; } ---------------------------- ============= On Linux ============= first fread failed after appending some text, fp1 is still eof buf: bar ============= On OS X ============= first fread failed after appending some text, fp1 is still eof second fread failed buf: Anyway, I think it's safe and preferable to clear EOF indicator in this case. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-24 04:28 Message: Logged In: YES user_id=835142 Originator: NO Hmm. It looks like glibc's fread will attempt reading even if EOF flag is already set. I've reproduced the problem on OS X 10.4.9 and Python 2.6a0 (trunk:54926M). This patch fixes it. ---------------------------------------------------------------------- Comment By: jos (josm) Date: 2007-04-23 23:27 Message: Logged In: YES user_id=1776568 Originator: YES I can easily reproduce this problem on OS X 10.4.9 using Python 2.4.3 and Python 2.5. Here's the code ################# import os, sys filename = "spam" f = open(filename, "w+") f.seek(0, 2) line = f.read() # EOF # Writing the same file using another fd f2 = open(filename, "a+") f2.write("Spam") f2.flush() f2.close() statinfo = os.stat(filename) print "file size: %d" % statinfo.st_size print "position : %d" % f.tell() line = f.read() # read() returns emtpy!! readlines?() works ok print "line : [%s]" % line ################# On my machine, it outputs the following ### file size: 4 position : 0 line : [] ### And just now I found that on ubuntu on the same machine (vmware), it works collect. ### file size: 4 position : 0 line : [Spam] ### My patched version python outputs the same result as above on OS X. We need clearerr() there, too because clearerr()'s job is not only only clearing error indicators but EOF indicators also. ---------------------------------------------------------------------- Comment By: Alexander Belopolsky (belopolsky) Date: 2007-04-23 20:22 Message: Logged In: YES user_id=835142 Originator: NO I was not able to reproduce the problem. I tried to read through the end of the file, append to the file from an external process and read again. It worked as expected: more data was read. The proposed patch seems to be redundant: once ferror(f->f_fp) is known to be 0, clearerr will do nothing and in the alternative branch clearerr is already there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1706039&group_id=5470 From noreply at sourceforge.net Sun Apr 29 14:57:59 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 29 Apr 2007 05:57:59 -0700 Subject: [Patches] [ python-Patches-1478364 ] Picky floats Message-ID: Patches item #1478364, was opened at 2006-04-28 14:33 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1478364&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: Picky floats Initial Comment: I came across a bug at work yesterday where I had written: if not delta: return 0.0 where delta was a floating point number. After a series of calculations piling up round-off error it took on a value on the order of 1e-8. Not zero, but it should have been. The fix was easy enough: if abs(delta) < EPSILON: return 0.0 for a suitable value of EPSILON. That got me to thinking... I'm sure I have plenty of other similar mistakes in my code. (Never was much of a numerical analysis guy...) What if there was a picky-float configure option that generated warnings if you compared a float with either another float or an int using "=="? Does that make sense to try for testing purposes? The initial implementation seemed straightforward enough, though I'm sure if this idea merits any serious consideration it will need some more work (at the least this behavior should probably be user-enabled). ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-29 14:57 Message: Logged In: YES user_id=21627 Originator: NO Since there were some objections to the patch, and apparently no progress in addressing these objections, I'm now rejecting the patch (on its first birthday). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2006-04-29 07:12 Message: Logged In: YES user_id=31435 The problem with looking "at some more floating point-intensive code (scipy?, MayaVi?)" is that people writing scientific code generally don't use float comparison incorrectly -- wading through false positives isn't all that much fun ;-) ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2006-04-29 01:43 Message: Logged In: YES user_id=44345 Yeah, I don't think configure-time selection is the ultimate choice either. Ideally, I think it's something you might want to enable on a per-module basis, but other than __future__ settings I'm not aware of anything. I just ran "make test" which runs everything twice. Looking at the output it seems to be complaining about 93 locations. All but 16 of them are in the test suite itself. I've not checked any of them carefully yet. A quick scan suggests most are comparisons with small integer-valued floats, so most are probably innocuous. I'm not surprised there's not a lot of problem in the standard library though, since it doesn't really do much in the way of numerical algorithms. It might be more instructive to look at some more floating point-intensive code (scipy?, MayaVi?). I'm less interested in finding specific bugs at this point though. I'm more concerned with deciding whether a suitably constrained implementation can be provided. (BTW, off-list I got a somewhat related report of lots of float comparison warnings in the Python C code by Intel's C compiler. I've asked him to run it again and file a bug report.) Skip ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-04-28 17:46 Message: Logged In: YES user_id=21627 I don't think it should be configure-time option. If implemented, it should be a run-time option, e.g. through a interpreter command line option, or even a flag in the sys module. OTOH, I doubt that it is very useful in practice. How many warnings do you get out of the test suite when this is on? ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2006-04-28 14:38 Message: Logged In: YES user_id=44345 FYI, regarding my original note and the attached patch, I realized that comparison with ints is mostly (always?) okay, so I took that out of the patch. Forgot to amend the wording above though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1478364&group_id=5470 From noreply at sourceforge.net Sun Apr 29 15:10:15 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 29 Apr 2007 06:10:15 -0700 Subject: [Patches] [ python-Patches-730473 ] Add Py_AtInit() startup hook for extenders Message-ID: Patches item #730473, was opened at 2003-05-01 00:26 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=730473&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Patrick Miller (patmiller) Assigned to: Nobody/Anonymous (nobody) Summary: Add Py_AtInit() startup hook for extenders Initial Comment: I work on several projects that have initialization requirements that need to grab control after Py_Initialize(), but before any user code runs (via input, script, -c, etc...). Note that these are Python clones that take advantage of an installed python (using its $prefix/lib/pythonx.x/*.py and site-packages/*) We could use PyImport_AppendInittab("sitecustomize",initsitecustomize); But if there already IS customization in sitecustomize.py, I've blown it away (and have to look it up and force an import). And if someone uses the -S flag, I'm screwed. I propose a hook styled after Py_AtExit(func) called Py_AtInit(func) which maintains a list of functions that are called in Py_Initialize right after main and site initializations. If the hook isn't used, then the cost is a single extra function call at initialization. Here's a spurious example: A customer wants a version of python that has all the math functions and his extensions to act like builtins... I would write (without refcnt or error checks ;-): #include "Python.h" static void after_init(void) { PyObject *builtin,*builtin_dict,*math,*math_dict,*user,*user_dict; builtin = PyImport_ImportModule("__builtin__"); builtin_dict = PyModule_GetDict(builtin); math = PyImport_ImportModule("math"); math_dict = PyModule_GetDict(math); user = PyImport_ImportModule("user"); user_dict = PyModule_GetDict(math); PyDict_Update(builtin_dictionary, math_dict); PyDict_Update(builtin_dictionary, user_dict); } int main(int argc, char** argv) { PyImport_AppendInittab("user",inituser); Py_AtInit(after_init); return Py_Main(argc, argv); } voila! An extended Python with new builtins. This is vastly better than hacking in through site.py or sitecustomize I actually want this to do some MPI initialization to setup a single user prompt with broadcast which has to run after Py_Initialize() but before the import of readline. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-29 15:10 Message: Logged In: YES user_id=21627 Originator: NO The patch looks good to me in principle. However, I wonder why you run the init functions after importing site; if you want it before any user-defined code, shouldn't you run it before site? Also, can you please provide documentation patches? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=730473&group_id=5470 From noreply at sourceforge.net Sun Apr 29 16:07:00 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 29 Apr 2007 07:07:00 -0700 Subject: [Patches] [ python-Patches-1709599 ] run test_1565150(test_os.py) only on NTFS Message-ID: Patches item #1709599, was opened at 2007-04-29 23:07 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709599&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Nobody/Anonymous (nobody) Summary: run test_1565150(test_os.py) only on NTFS Initial Comment: http://www.nabble.com/Python-2.5.1-tf3664463.html#a10239786 Maybe this patch helps. This patch gets file system name via GetVolumeInformationA and runs test only on NTFS. Thank you. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709599&group_id=5470 From noreply at sourceforge.net Sun Apr 29 16:54:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 29 Apr 2007 07:54:12 -0700 Subject: [Patches] [ python-Patches-730473 ] Add Py_AtInit() startup hook for extenders Message-ID: Patches item #730473, was opened at 2003-04-30 15:26 Message generated for change (Comment added) made by patmiller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=730473&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Patrick Miller (patmiller) Assigned to: Nobody/Anonymous (nobody) Summary: Add Py_AtInit() startup hook for extenders Initial Comment: I work on several projects that have initialization requirements that need to grab control after Py_Initialize(), but before any user code runs (via input, script, -c, etc...). Note that these are Python clones that take advantage of an installed python (using its $prefix/lib/pythonx.x/*.py and site-packages/*) We could use PyImport_AppendInittab("sitecustomize",initsitecustomize); But if there already IS customization in sitecustomize.py, I've blown it away (and have to look it up and force an import). And if someone uses the -S flag, I'm screwed. I propose a hook styled after Py_AtExit(func) called Py_AtInit(func) which maintains a list of functions that are called in Py_Initialize right after main and site initializations. If the hook isn't used, then the cost is a single extra function call at initialization. Here's a spurious example: A customer wants a version of python that has all the math functions and his extensions to act like builtins... I would write (without refcnt or error checks ;-): #include "Python.h" static void after_init(void) { PyObject *builtin,*builtin_dict,*math,*math_dict,*user,*user_dict; builtin = PyImport_ImportModule("__builtin__"); builtin_dict = PyModule_GetDict(builtin); math = PyImport_ImportModule("math"); math_dict = PyModule_GetDict(math); user = PyImport_ImportModule("user"); user_dict = PyModule_GetDict(math); PyDict_Update(builtin_dictionary, math_dict); PyDict_Update(builtin_dictionary, user_dict); } int main(int argc, char** argv) { PyImport_AppendInittab("user",inituser); Py_AtInit(after_init); return Py_Main(argc, argv); } voila! An extended Python with new builtins. This is vastly better than hacking in through site.py or sitecustomize I actually want this to do some MPI initialization to setup a single user prompt with broadcast which has to run after Py_Initialize() but before the import of readline. ---------------------------------------------------------------------- >Comment By: Patrick Miller (patmiller) Date: 2007-04-29 07:54 Message: Logged In: YES user_id=30074 Originator: YES Martin, You are absolutely right, the call to initinitialize() should occur before initsite(). I'll prepare a new patch with that change and also include doc updates. Pat ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-29 06:10 Message: Logged In: YES user_id=21627 Originator: NO The patch looks good to me in principle. However, I wonder why you run the init functions after importing site; if you want it before any user-defined code, shouldn't you run it before site? Also, can you please provide documentation patches? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=730473&group_id=5470 From noreply at sourceforge.net Mon Apr 30 03:52:32 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 29 Apr 2007 18:52:32 -0700 Subject: [Patches] [ python-Patches-1709112 ] test_1686475 of test_os & pagefile.sys Message-ID: Patches item #1709112, was opened at 2007-04-28 06:35 Message generated for change (Comment added) made by abkhd You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: A.B., Khalid (abkhd) Assigned to: Nobody/Anonymous (nobody) Summary: test_1686475 of test_os & pagefile.sys Initial Comment: test_1686475 of test_os appears to rely on the existence of "pagefile.sys" on the C drive like so: def test_1686475(self): # Verify that an open file can be stat'ed try: os.stat(r"c:\pagefile.sys") except WindowsError, e: if e == 2: # file does not exist; cannot run test return self.fail("Could not stat pagefile.sys") But since pagefile.sys may not be on every C drive and since the exception object e is not a numeric then that test is probably going to fail if the pagefile.sys does not exist where it is expected. In such a case Windows returns "[Error 5] Access is denied: 'c:\\pagefile.sys'". So we need to test if e.winerror == 5 to skip this and return without failing. Patch "test_os.patch" is attached. By the way, I see the same problem in the trunk, so maybe this need to be applied there as well. ---------------------------------------------------------------------- >Comment By: A.B., Khalid (abkhd) Date: 2007-04-30 01:52 Message: Logged In: YES user_id=1079026 Originator: YES Please see http://www.python.org/sf/1686475 for reference. Martin, I think you should be checking for ERROR_ACCESS_DENIED not ERROR_SHARING_VIOLATION as you do in your patch to r54685 and r54686. The attached posixmodule patch fixes this issue for me on MinGW. Can you try it out? File Added: test_1686475.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 From noreply at sourceforge.net Mon Apr 30 07:16:27 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 29 Apr 2007 22:16:27 -0700 Subject: [Patches] [ python-Patches-1709112 ] test_1686475 of test_os & pagefile.sys Message-ID: Patches item #1709112, was opened at 2007-04-28 08:35 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: A.B., Khalid (abkhd) Assigned to: Nobody/Anonymous (nobody) Summary: test_1686475 of test_os & pagefile.sys Initial Comment: test_1686475 of test_os appears to rely on the existence of "pagefile.sys" on the C drive like so: def test_1686475(self): # Verify that an open file can be stat'ed try: os.stat(r"c:\pagefile.sys") except WindowsError, e: if e == 2: # file does not exist; cannot run test return self.fail("Could not stat pagefile.sys") But since pagefile.sys may not be on every C drive and since the exception object e is not a numeric then that test is probably going to fail if the pagefile.sys does not exist where it is expected. In such a case Windows returns "[Error 5] Access is denied: 'c:\\pagefile.sys'". So we need to test if e.winerror == 5 to skip this and return without failing. Patch "test_os.patch" is attached. By the way, I see the same problem in the trunk, so maybe this need to be applied there as well. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2007-04-30 07:16 Message: Logged In: YES user_id=21627 Originator: NO Why should I not check for ERROR_SHARING_VIOLATION? The OP complains that he sees an error "The process cannot access the file because it is being used by another process" in 2.5.0, and can stat the file in 2.4. So it is clearly ERROR_SHARING_VIOLATION that is being returned (also on the system on which I developed the patch). ---------------------------------------------------------------------- Comment By: A.B., Khalid (abkhd) Date: 2007-04-30 03:52 Message: Logged In: YES user_id=1079026 Originator: YES Please see http://www.python.org/sf/1686475 for reference. Martin, I think you should be checking for ERROR_ACCESS_DENIED not ERROR_SHARING_VIOLATION as you do in your patch to r54685 and r54686. The attached posixmodule patch fixes this issue for me on MinGW. Can you try it out? File Added: test_1686475.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 From noreply at sourceforge.net Mon Apr 30 11:07:10 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 30 Apr 2007 02:07:10 -0700 Subject: [Patches] [ python-Patches-1709112 ] test_1686475 of test_os & pagefile.sys Message-ID: Patches item #1709112, was opened at 2007-04-28 06:35 Message generated for change (Comment added) made by abkhd You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: A.B., Khalid (abkhd) Assigned to: Nobody/Anonymous (nobody) Summary: test_1686475 of test_os & pagefile.sys Initial Comment: test_1686475 of test_os appears to rely on the existence of "pagefile.sys" on the C drive like so: def test_1686475(self): # Verify that an open file can be stat'ed try: os.stat(r"c:\pagefile.sys") except WindowsError, e: if e == 2: # file does not exist; cannot run test return self.fail("Could not stat pagefile.sys") But since pagefile.sys may not be on every C drive and since the exception object e is not a numeric then that test is probably going to fail if the pagefile.sys does not exist where it is expected. In such a case Windows returns "[Error 5] Access is denied: 'c:\\pagefile.sys'". So we need to test if e.winerror == 5 to skip this and return without failing. Patch "test_os.patch" is attached. By the way, I see the same problem in the trunk, so maybe this need to be applied there as well. ---------------------------------------------------------------------- >Comment By: A.B., Khalid (abkhd) Date: 2007-04-30 09:07 Message: Logged In: YES user_id=1079026 Originator: YES I had thought that the file was opened exclusively by Windows and so the latter denied access to it (including sharing). However, it might be worth a while to note how others handle this problem. Please see ntfs.c of MIT/GNU Scheme: http://www.koders.com/c/fid3A61BD3B46FA62F5E9F99E5860DCAE5BC2B7FA2C.aspx and ntfs.h of the same project: http://www.koders.com/c/fid5FA9A6766ABABAAB4DC09266CA61866F8270A139.aspx They actually test for both errors in addtion to ERROR_DRIVE_LOCKED to determine if stat has worked or not. Regards, Khalid More references: http://www.supportnet.de/stat/2002/7/id98848.asp http://www.cygwin.com/ml/cygwin/2000-01/msg00274.html http://ftp.gnu.org/gnu/mit-scheme/stable.pkg/7.7.1/changelog.txt ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-30 05:16 Message: Logged In: YES user_id=21627 Originator: NO Why should I not check for ERROR_SHARING_VIOLATION? The OP complains that he sees an error "The process cannot access the file because it is being used by another process" in 2.5.0, and can stat the file in 2.4. So it is clearly ERROR_SHARING_VIOLATION that is being returned (also on the system on which I developed the patch). ---------------------------------------------------------------------- Comment By: A.B., Khalid (abkhd) Date: 2007-04-30 01:52 Message: Logged In: YES user_id=1079026 Originator: YES Please see http://www.python.org/sf/1686475 for reference. Martin, I think you should be checking for ERROR_ACCESS_DENIED not ERROR_SHARING_VIOLATION as you do in your patch to r54685 and r54686. The attached posixmodule patch fixes this issue for me on MinGW. Can you try it out? File Added: test_1686475.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 From noreply at sourceforge.net Mon Apr 30 12:01:12 2007 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 30 Apr 2007 03:01:12 -0700 Subject: [Patches] [ python-Patches-1709112 ] test_1686475 of test_os & pagefile.sys Message-ID: Patches item #1709112, was opened at 2007-04-28 06:35 Message generated for change (Comment added) made by abkhd You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Tests Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: A.B., Khalid (abkhd) Assigned to: Nobody/Anonymous (nobody) Summary: test_1686475 of test_os & pagefile.sys Initial Comment: test_1686475 of test_os appears to rely on the existence of "pagefile.sys" on the C drive like so: def test_1686475(self): # Verify that an open file can be stat'ed try: os.stat(r"c:\pagefile.sys") except WindowsError, e: if e == 2: # file does not exist; cannot run test return self.fail("Could not stat pagefile.sys") But since pagefile.sys may not be on every C drive and since the exception object e is not a numeric then that test is probably going to fail if the pagefile.sys does not exist where it is expected. In such a case Windows returns "[Error 5] Access is denied: 'c:\\pagefile.sys'". So we need to test if e.winerror == 5 to skip this and return without failing. Patch "test_os.patch" is attached. By the way, I see the same problem in the trunk, so maybe this need to be applied there as well. ---------------------------------------------------------------------- >Comment By: A.B., Khalid (abkhd) Date: 2007-04-30 10:01 Message: Logged In: YES user_id=1079026 Originator: YES Another source seems to suggest that the error message will differ depending on the Windows version. I think the error message might be more related to the underlying file system. Hence since my C drive is formated as FAT WinXP might be reporting attempted access to the locked pagefile.sys as ERROR_ACCESS_DENIED, however, in other machines where the C drive could be formated as an NTFS then the error message will be changed to SHARING_ERROR_VIOLATION. This makes sense because pagefile.sys seems to be opened exclusively by Windows in either case. The error differs depending on the capability of the file system. And so one would be expected to test for both errors to see if stat worked. Regards, Khalid Reference [Search of ERROR_SHARING_VIOLATION] in syscalls.cc of Cygwin: http://www.koders.com/cpp/fid05B16888E9827D2766C45C718ED8620BC82B4DCE.aspx pagefile.sys ---------------------------------------------------------------------- Comment By: A.B., Khalid (abkhd) Date: 2007-04-30 09:07 Message: Logged In: YES user_id=1079026 Originator: YES I had thought that the file was opened exclusively by Windows and so the latter denied access to it (including sharing). However, it might be worth a while to note how others handle this problem. Please see ntfs.c of MIT/GNU Scheme: http://www.koders.com/c/fid3A61BD3B46FA62F5E9F99E5860DCAE5BC2B7FA2C.aspx and ntfs.h of the same project: http://www.koders.com/c/fid5FA9A6766ABABAAB4DC09266CA61866F8270A139.aspx They actually test for both errors in addtion to ERROR_DRIVE_LOCKED to determine if stat has worked or not. Regards, Khalid More references: http://www.supportnet.de/stat/2002/7/id98848.asp http://www.cygwin.com/ml/cygwin/2000-01/msg00274.html http://ftp.gnu.org/gnu/mit-scheme/stable.pkg/7.7.1/changelog.txt ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2007-04-30 05:16 Message: Logged In: YES user_id=21627 Originator: NO Why should I not check for ERROR_SHARING_VIOLATION? The OP complains that he sees an error "The process cannot access the file because it is being used by another process" in 2.5.0, and can stat the file in 2.4. So it is clearly ERROR_SHARING_VIOLATION that is being returned (also on the system on which I developed the patch). ---------------------------------------------------------------------- Comment By: A.B., Khalid (abkhd) Date: 2007-04-30 01:52 Message: Logged In: YES user_id=1079026 Originator: YES Please see http://www.python.org/sf/1686475 for reference. Martin, I think you should be checking for ERROR_ACCESS_DENIED not ERROR_SHARING_VIOLATION as you do in your patch to r54685 and r54686. The attached posixmodule patch fixes this issue for me on MinGW. Can you try it out? File Added: test_1686475.patch ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1709112&group_id=5470