From suannegirest at bushemail.com Mon Mar 1 12:15:32 2004 From: suannegirest at bushemail.com (Rita Hairston) Date: Sun Feb 29 20:08:13 2004 Subject: [Patches] 73.NQI. tree trunk member.. Message-ID: drop the hammer on the next girl you screw... http://emarkdinbs.com/vp5 No more of this sort of material. Honoured in 24-48 hours. http://dietetic.amilsdcx.com/a.html rascal beatific distillate wallow aristotle dispersion brigade dieldrin dusseldorf uniplex coliform cosmetic sculpt xH From noreply at sourceforge.net Mon Mar 1 04:27:45 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 1 04:28:04 2004 Subject: [Patches] [ python-Patches-906702 ] Syntax-related improvements to IDLE Message-ID: Patches item #906702, was opened at 2004-02-28 20:40 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=906702&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Noam Raphael (noamr) >Assigned to: Kurt B. Kaiser (kbk) Summary: Syntax-related improvements to IDLE Initial Comment: Yes! It's finally here! The big bunch of syntax-related improvements to IDLE! Here are a few highlights: * Completion in Visual C++ style! * CallTip windows now disappear when you don't want them! * ParenMatch now works! And in a more general way: * The MultiCall class enables one event to trigger multiple actions, so that you extensions can do whatever they want without fighting who'll know when a key is being released (for example). * The HyperParser class gives extensions a better understanding of Python code, so that CallTips and ParenMatch can now be shown from anywhere in the code (not only after opening/closing parens), and CallTips and AutoCompletion is available for complex expressions, not just for expressions which look like identifier1.identifer2. IDLE with these changes was tested by about 15 people for a few months, and beside some minor inconviniences (for example, concerning when AutoComplete chooses to complete), no critical bugs were found. Here's a complete description of all the changes to files: * Used MultiCall In order to allow multiple actions to happen when one event happens (for example, when closing a bracket, show the opening bracket and close the calltip), I wrote a class called MultiCall, which, in this case, inherits from the Text widget, and overrides binding functions, so that all matching functions will be called when an event takes place. New file: MultiCall.py EditorWindow.py: diff 1, line 8: importing MultiCall diff 2, line 89: Use MultiCallCreator(Text) instead of Text diff 3, line 226: bind set_line_and_column using virtual events, so that it will be handled by MultiCall diff 4, line 333: Control+C, in Windows keys, means both copy and break. The three lines added solve the conflict. diff 5,6,7, line 524: When changing key configuration, first the current keys must be unbinded, then the new ones binded. configDialog.py: diffs 1,2,3: When changing key configuration, first the current keys must be unbinded, then the new ones binded. * Improved parsing abilities - HyperParser The new HyperParser class uses PyParse to find information about the bracketing structure of a Python code, and to find the expression which should be evaluated in CallTips and AutoComplete. New file: HyperParser.py EditorWindow.py: diff 8, 9, line 1072: Previously, the prompt was detected textually, by searching for ">>>". This is not generic, and I changed it to look for a text tagged by the "console" tag. PyParse.py: diff 1, line 16: 'if' and 'for' can't be considered block openers now, since list comprehension was introduced. diff 2-5, line 147: using the "prompt" tag instead of searching for ">>>". diff 6-15, line 357: Now, _study2 also saves information about the bracketing structure of the code. PyShell.py: diff 1, line 1049: use the "prompt" tag instead of searching for ">>>" * Added the AutoComplete extension: This extension opens a completion window after typing a dot or when requesting a completion using tab or control+space. It will evaluate code with function calls only when requested by control+space. New file: AutoComplete.py New file: AutoCompleteWindow.py config-extensions.def: diff 5: three new bindings - force-open-completions opens a completion win and is ready to evaluate functions. autocomplete tries to complete the current word, and if fails, opens a completion window (happens on tab). try-open-completions will open a completion window when no functions calls need to be done. run.py: diffs 1,2,3: Added functions in run.py for fetching the available completions, in a method copied from the method CallTips used when fetching the call tip. * Improved the CallTips extension: This extension is greatly improved, by using HyperParser: CallTips now work for multi-lined function calls, they can be opened from anywhere inside a function brackets, and can evaluate expressions which result in functions. CallTips.py: diff 1, line 6: The future plans are already here. diff 2, line 11: no need to import string diff 3, line 14: import HyperParser diff 4, line 20: Now there's a menu command to open a calltip. diff 5, line 39: cosmetical diff 6-11, line 44: Now there's no paren_open_event and paren_close_event, since you can open CallTips without typing parens. There's force_open_calltip_event, which opens a calltip and is ready to make function calls, and try_open_calltip_event, which will not agree to do such a thing. We also use HyperParser, instead of finding the function by ourselves. diff 12, line 153: It makes more sense to show the repr of default arguments in calltips, not the str of them. CallTipWindow.py: Now, the CallTipWindow is smart - it binds events to hide itself and to position itself. Hiding occurs when the cursor gets out of the parens. config-extensions.def: diffs 1,2, line 42: We have three new bindings: force-open-calltip opens a calltip and make function calls, if needed. try-open-calltip opens a calltip if it doesn't need to make function calls. refresh-calltip will close a calltip if not needed, and change it to another one if needed. It is binded both to parenright and 0, because if the shift was raised before the zero/parenright key, the event generated is KeyRelease-0. * Improved the ParenMatch extension: First, it now works, thanks to MultiCall. Also, it now can be invoked without closing a paren, and the surrounding parens will be highlighted. ParenMatch.py: All the code for event handling and finding the paren area was rewritten, using HyperParser, and relying on MultiCall. config-extensions.def: diff 3, line 50: enable it again diff 4,5: changed the binding so we now have flash-paren, which shows the surrounding parens, and paren-closed, which is instead of flash-open-paren. Patch and enjoy! Noam Raphael ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=906702&group_id=5470 From noreply at sourceforge.net Mon Mar 1 04:35:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 1 04:35:12 2004 Subject: [Patches] [ python-Patches-907403 ] Improvements to cStringIO.writelines() Message-ID: Patches item #907403, was opened at 2004-03-01 04:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=907403&group_id=5470 Category: Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Improvements to cStringIO.writelines() Initial Comment: Currently, cStringIO.writelines() only accepts a sequence argument and it passes that argument internally to str.join() to construct a large temporary string before writing. The patch allows the method to accept any iterable argument and for the lines to be written one at a time which cuts memory utilization in half. The result is especially valuable when used with generators or itertools because the strings are constructed on the fly rather than all at once. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=907403&group_id=5470 From noreply at sourceforge.net Mon Mar 1 07:20:53 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 1 07:21:01 2004 Subject: [Patches] [ python-Patches-876206 ] scary frame speed hacks Message-ID: Patches item #876206, was opened at 2004-01-13 11:49 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hudson (mwh) Assigned to: Jeremy Hylton (jhylton) Summary: scary frame speed hacks Initial Comment: In ceval.c we find /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ This patch takes that idea rather further than you might have expected... it creates a "light" subtype of frame that assumes certain things about the frame, gives this type its own free list (so it can assume more about objects on the freelist) and converts light frames into "heavy" frames when assumptions stop being true. Good for a ~5% improvement on "./python -s 'def f(): pass' 'f()'"; a bit less on pystone. It also conflicts slightly with my function reorg patch -- apply that first, apply this, ignore the reject and edit func_caller_nofrees in funcobject.c to call PyFrame_NewLight. All three patches I just submitted together get ~6% on pystone. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 07:20 Message: Logged In: YES user_id=80475 Armin's second patch gives gives the expected speedups on a Pentium3 running WinME, and the test suite runs without exception. I recommend accepting and applying this patch as is. Further improvements can be considered separately. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 08:04 Message: Logged In: YES user_id=4771 I guess the idea was just in the air, after your published attempts. Ideally I'd have liked to have the cached frame depend on the globals as well as the code object itself; I considered moving the cache field to function objects. This way you also save the f_globals and f_builtins initialization. There were problems but maybe we should try harder. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-02-02 06:20 Message: Logged In: YES user_id=6656 Did I mention this idea to you or did you come up with it independently? I forget... I'll try to time stuff on my iBook tomorrow. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 12:03 Message: Logged In: YES user_id=4771 Here is yet another try, which seems to perform better on my PentiumIII. I get the following speed improvements for this patch alone, for a loop calling an empty function: zombie-frames.diff: 11.4% (PyStone 3.8%) scary-frame-hacks.diff: 6.4% (PyStone 0.85%) The idea is to get rid of the free_list and instead store the most recently finished ("zombie") frame in an internal field of the code object. This saves half of the frame creation overhead because half of the fields are already correct when the frame is reused, e.g. f_code, f_nlocals, f_stacksize, f_valuestack... (you might need to cvs up frameobject.c before you can apply the patch) ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-15 06:02 Message: Logged In: YES user_id=6656 I'm fairly sure this made a difference on my iBook; haven't tried on x86. It's possible that the correct response to this patch is to add "... nah, not worth it" to the XXX comment in ceval.c... ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-14 12:35 Message: Logged In: YES user_id=4771 (Side note first: I'm not sure 'builtins = back->f_builtins' is right.) Is the whole subclassing complexity worth the effort, given that the invariants of light frames only seem to be that four specific fields are null? Changing the type of an object under Python code's feet is calling for troubles. Moreover it is bound to break code that expect 'type(frame)==FrameType', even if such code can be considered bad style. Moreover it requires a number of hacks here and there -- e.g. you turn a light frame into a "heavy" frame when f_trace is set; is it on purpose that you don't do it when f_locals is set? I cannot seem to get reliable performance results on my machine, but maybe you want to compare with the attached patch which speeds up the regular PyFrame_New by putting stronger invariants on all the frames in the free_list. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-13 13:23 Message: Logged In: YES user_id=6656 sigh ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2004-01-13 13:20 Message: Logged In: YES user_id=31392 I don't see any files attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 From noreply at sourceforge.net Mon Mar 1 07:23:43 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 1 07:23:49 2004 Subject: [Patches] [ python-Patches-876206 ] scary frame speed hacks Message-ID: Patches item #876206, was opened at 2004-01-13 16:49 Message generated for change (Comment added) made by mwh You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hudson (mwh) Assigned to: Jeremy Hylton (jhylton) Summary: scary frame speed hacks Initial Comment: In ceval.c we find /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ This patch takes that idea rather further than you might have expected... it creates a "light" subtype of frame that assumes certain things about the frame, gives this type its own free list (so it can assume more about objects on the freelist) and converts light frames into "heavy" frames when assumptions stop being true. Good for a ~5% improvement on "./python -s 'def f(): pass' 'f()'"; a bit less on pystone. It also conflicts slightly with my function reorg patch -- apply that first, apply this, ignore the reject and edit func_caller_nofrees in funcobject.c to call PyFrame_NewLight. All three patches I just submitted together get ~6% on pystone. ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2004-03-01 12:23 Message: Logged In: YES user_id=6656 It slows recursive functions down a noticeable amount (did this get noted anywhere? Maybe Armin & I just talked about it on IRC), so that should be considered before this patch is applied. But I think it's probably worth it, FWIW. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 12:20 Message: Logged In: YES user_id=80475 Armin's second patch gives gives the expected speedups on a Pentium3 running WinME, and the test suite runs without exception. I recommend accepting and applying this patch as is. Further improvements can be considered separately. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 13:04 Message: Logged In: YES user_id=4771 I guess the idea was just in the air, after your published attempts. Ideally I'd have liked to have the cached frame depend on the globals as well as the code object itself; I considered moving the cache field to function objects. This way you also save the f_globals and f_builtins initialization. There were problems but maybe we should try harder. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-02-02 11:20 Message: Logged In: YES user_id=6656 Did I mention this idea to you or did you come up with it independently? I forget... I'll try to time stuff on my iBook tomorrow. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 17:03 Message: Logged In: YES user_id=4771 Here is yet another try, which seems to perform better on my PentiumIII. I get the following speed improvements for this patch alone, for a loop calling an empty function: zombie-frames.diff: 11.4% (PyStone 3.8%) scary-frame-hacks.diff: 6.4% (PyStone 0.85%) The idea is to get rid of the free_list and instead store the most recently finished ("zombie") frame in an internal field of the code object. This saves half of the frame creation overhead because half of the fields are already correct when the frame is reused, e.g. f_code, f_nlocals, f_stacksize, f_valuestack... (you might need to cvs up frameobject.c before you can apply the patch) ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-15 11:02 Message: Logged In: YES user_id=6656 I'm fairly sure this made a difference on my iBook; haven't tried on x86. It's possible that the correct response to this patch is to add "... nah, not worth it" to the XXX comment in ceval.c... ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-14 17:35 Message: Logged In: YES user_id=4771 (Side note first: I'm not sure 'builtins = back->f_builtins' is right.) Is the whole subclassing complexity worth the effort, given that the invariants of light frames only seem to be that four specific fields are null? Changing the type of an object under Python code's feet is calling for troubles. Moreover it is bound to break code that expect 'type(frame)==FrameType', even if such code can be considered bad style. Moreover it requires a number of hacks here and there -- e.g. you turn a light frame into a "heavy" frame when f_trace is set; is it on purpose that you don't do it when f_locals is set? I cannot seem to get reliable performance results on my machine, but maybe you want to compare with the attached patch which speeds up the regular PyFrame_New by putting stronger invariants on all the frames in the free_list. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-13 18:23 Message: Logged In: YES user_id=6656 sigh ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2004-01-13 18:20 Message: Logged In: YES user_id=31392 I don't see any files attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 From noreply at sourceforge.net Mon Mar 1 08:25:05 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 1 08:25:13 2004 Subject: [Patches] [ python-Patches-876206 ] scary frame speed hacks Message-ID: Patches item #876206, was opened at 2004-01-13 11:49 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hudson (mwh) Assigned to: Jeremy Hylton (jhylton) Summary: scary frame speed hacks Initial Comment: In ceval.c we find /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ This patch takes that idea rather further than you might have expected... it creates a "light" subtype of frame that assumes certain things about the frame, gives this type its own free list (so it can assume more about objects on the freelist) and converts light frames into "heavy" frames when assumptions stop being true. Good for a ~5% improvement on "./python -s 'def f(): pass' 'f()'"; a bit less on pystone. It also conflicts slightly with my function reorg patch -- apply that first, apply this, ignore the reject and edit func_caller_nofrees in funcobject.c to call PyFrame_NewLight. All three patches I just submitted together get ~6% on pystone. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 08:25 Message: Logged In: YES user_id=80475 The effect on recursive functions could be mitigated by restoring the freelist and falling back to it when code->co_zombieframe == NULL. I don't know if that is worth it. The current patch is a code simplication as well as an optimization. Adding back the freelist, adds a lot of clutter. Python is not especially friendly to recursive functions anyway. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-03-01 07:23 Message: Logged In: YES user_id=6656 It slows recursive functions down a noticeable amount (did this get noted anywhere? Maybe Armin & I just talked about it on IRC), so that should be considered before this patch is applied. But I think it's probably worth it, FWIW. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 07:20 Message: Logged In: YES user_id=80475 Armin's second patch gives gives the expected speedups on a Pentium3 running WinME, and the test suite runs without exception. I recommend accepting and applying this patch as is. Further improvements can be considered separately. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 08:04 Message: Logged In: YES user_id=4771 I guess the idea was just in the air, after your published attempts. Ideally I'd have liked to have the cached frame depend on the globals as well as the code object itself; I considered moving the cache field to function objects. This way you also save the f_globals and f_builtins initialization. There were problems but maybe we should try harder. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-02-02 06:20 Message: Logged In: YES user_id=6656 Did I mention this idea to you or did you come up with it independently? I forget... I'll try to time stuff on my iBook tomorrow. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 12:03 Message: Logged In: YES user_id=4771 Here is yet another try, which seems to perform better on my PentiumIII. I get the following speed improvements for this patch alone, for a loop calling an empty function: zombie-frames.diff: 11.4% (PyStone 3.8%) scary-frame-hacks.diff: 6.4% (PyStone 0.85%) The idea is to get rid of the free_list and instead store the most recently finished ("zombie") frame in an internal field of the code object. This saves half of the frame creation overhead because half of the fields are already correct when the frame is reused, e.g. f_code, f_nlocals, f_stacksize, f_valuestack... (you might need to cvs up frameobject.c before you can apply the patch) ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-15 06:02 Message: Logged In: YES user_id=6656 I'm fairly sure this made a difference on my iBook; haven't tried on x86. It's possible that the correct response to this patch is to add "... nah, not worth it" to the XXX comment in ceval.c... ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-14 12:35 Message: Logged In: YES user_id=4771 (Side note first: I'm not sure 'builtins = back->f_builtins' is right.) Is the whole subclassing complexity worth the effort, given that the invariants of light frames only seem to be that four specific fields are null? Changing the type of an object under Python code's feet is calling for troubles. Moreover it is bound to break code that expect 'type(frame)==FrameType', even if such code can be considered bad style. Moreover it requires a number of hacks here and there -- e.g. you turn a light frame into a "heavy" frame when f_trace is set; is it on purpose that you don't do it when f_locals is set? I cannot seem to get reliable performance results on my machine, but maybe you want to compare with the attached patch which speeds up the regular PyFrame_New by putting stronger invariants on all the frames in the free_list. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-13 13:23 Message: Logged In: YES user_id=6656 sigh ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2004-01-13 13:20 Message: Logged In: YES user_id=31392 I don't see any files attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 From noreply at sourceforge.net Mon Mar 1 08:59:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 1 08:59:33 2004 Subject: [Patches] [ python-Patches-876206 ] scary frame speed hacks Message-ID: Patches item #876206, was opened at 2004-01-13 16:49 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hudson (mwh) Assigned to: Jeremy Hylton (jhylton) Summary: scary frame speed hacks Initial Comment: In ceval.c we find /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ This patch takes that idea rather further than you might have expected... it creates a "light" subtype of frame that assumes certain things about the frame, gives this type its own free list (so it can assume more about objects on the freelist) and converts light frames into "heavy" frames when assumptions stop being true. Good for a ~5% improvement on "./python -s 'def f(): pass' 'f()'"; a bit less on pystone. It also conflicts slightly with my function reorg patch -- apply that first, apply this, ignore the reject and edit func_caller_nofrees in funcobject.c to call PyFrame_NewLight. All three patches I just submitted together get ~6% on pystone. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2004-03-01 13:59 Message: Logged In: YES user_id=4771 On a small recursive example it slows down from 2.64s to 3.26s. This is a serious difference (20%). Is it bad enough to keep the freelist ? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 13:25 Message: Logged In: YES user_id=80475 The effect on recursive functions could be mitigated by restoring the freelist and falling back to it when code->co_zombieframe == NULL. I don't know if that is worth it. The current patch is a code simplication as well as an optimization. Adding back the freelist, adds a lot of clutter. Python is not especially friendly to recursive functions anyway. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-03-01 12:23 Message: Logged In: YES user_id=6656 It slows recursive functions down a noticeable amount (did this get noted anywhere? Maybe Armin & I just talked about it on IRC), so that should be considered before this patch is applied. But I think it's probably worth it, FWIW. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 12:20 Message: Logged In: YES user_id=80475 Armin's second patch gives gives the expected speedups on a Pentium3 running WinME, and the test suite runs without exception. I recommend accepting and applying this patch as is. Further improvements can be considered separately. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 13:04 Message: Logged In: YES user_id=4771 I guess the idea was just in the air, after your published attempts. Ideally I'd have liked to have the cached frame depend on the globals as well as the code object itself; I considered moving the cache field to function objects. This way you also save the f_globals and f_builtins initialization. There were problems but maybe we should try harder. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-02-02 11:20 Message: Logged In: YES user_id=6656 Did I mention this idea to you or did you come up with it independently? I forget... I'll try to time stuff on my iBook tomorrow. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 17:03 Message: Logged In: YES user_id=4771 Here is yet another try, which seems to perform better on my PentiumIII. I get the following speed improvements for this patch alone, for a loop calling an empty function: zombie-frames.diff: 11.4% (PyStone 3.8%) scary-frame-hacks.diff: 6.4% (PyStone 0.85%) The idea is to get rid of the free_list and instead store the most recently finished ("zombie") frame in an internal field of the code object. This saves half of the frame creation overhead because half of the fields are already correct when the frame is reused, e.g. f_code, f_nlocals, f_stacksize, f_valuestack... (you might need to cvs up frameobject.c before you can apply the patch) ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-15 11:02 Message: Logged In: YES user_id=6656 I'm fairly sure this made a difference on my iBook; haven't tried on x86. It's possible that the correct response to this patch is to add "... nah, not worth it" to the XXX comment in ceval.c... ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-14 17:35 Message: Logged In: YES user_id=4771 (Side note first: I'm not sure 'builtins = back->f_builtins' is right.) Is the whole subclassing complexity worth the effort, given that the invariants of light frames only seem to be that four specific fields are null? Changing the type of an object under Python code's feet is calling for troubles. Moreover it is bound to break code that expect 'type(frame)==FrameType', even if such code can be considered bad style. Moreover it requires a number of hacks here and there -- e.g. you turn a light frame into a "heavy" frame when f_trace is set; is it on purpose that you don't do it when f_locals is set? I cannot seem to get reliable performance results on my machine, but maybe you want to compare with the attached patch which speeds up the regular PyFrame_New by putting stronger invariants on all the frames in the free_list. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-13 18:23 Message: Logged In: YES user_id=6656 sigh ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2004-01-13 18:20 Message: Logged In: YES user_id=31392 I don't see any files attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 From noreply at sourceforge.net Mon Mar 1 10:50:09 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 1 10:50:19 2004 Subject: [Patches] [ python-Patches-876206 ] scary frame speed hacks Message-ID: Patches item #876206, was opened at 2004-01-13 11:49 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Michael Hudson (mwh) Assigned to: Jeremy Hylton (jhylton) Summary: scary frame speed hacks Initial Comment: In ceval.c we find /* XXX Perhaps we should create a specialized PyFrame_New() that doesn't take locals, but does take builtins without sanity checking them. */ This patch takes that idea rather further than you might have expected... it creates a "light" subtype of frame that assumes certain things about the frame, gives this type its own free list (so it can assume more about objects on the freelist) and converts light frames into "heavy" frames when assumptions stop being true. Good for a ~5% improvement on "./python -s 'def f(): pass' 'f()'"; a bit less on pystone. It also conflicts slightly with my function reorg patch -- apply that first, apply this, ignore the reject and edit func_caller_nofrees in funcobject.c to call PyFrame_NewLight. All three patches I just submitted together get ~6% on pystone. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 10:50 Message: Logged In: YES user_id=80475 I think that's a question for python-dev. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-01 08:59 Message: Logged In: YES user_id=4771 On a small recursive example it slows down from 2.64s to 3.26s. This is a serious difference (20%). Is it bad enough to keep the freelist ? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 08:25 Message: Logged In: YES user_id=80475 The effect on recursive functions could be mitigated by restoring the freelist and falling back to it when code->co_zombieframe == NULL. I don't know if that is worth it. The current patch is a code simplication as well as an optimization. Adding back the freelist, adds a lot of clutter. Python is not especially friendly to recursive functions anyway. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-03-01 07:23 Message: Logged In: YES user_id=6656 It slows recursive functions down a noticeable amount (did this get noted anywhere? Maybe Armin & I just talked about it on IRC), so that should be considered before this patch is applied. But I think it's probably worth it, FWIW. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-01 07:20 Message: Logged In: YES user_id=80475 Armin's second patch gives gives the expected speedups on a Pentium3 running WinME, and the test suite runs without exception. I recommend accepting and applying this patch as is. Further improvements can be considered separately. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 08:04 Message: Logged In: YES user_id=4771 I guess the idea was just in the air, after your published attempts. Ideally I'd have liked to have the cached frame depend on the globals as well as the code object itself; I considered moving the cache field to function objects. This way you also save the f_globals and f_builtins initialization. There were problems but maybe we should try harder. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-02-02 06:20 Message: Logged In: YES user_id=6656 Did I mention this idea to you or did you come up with it independently? I forget... I'll try to time stuff on my iBook tomorrow. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 12:03 Message: Logged In: YES user_id=4771 Here is yet another try, which seems to perform better on my PentiumIII. I get the following speed improvements for this patch alone, for a loop calling an empty function: zombie-frames.diff: 11.4% (PyStone 3.8%) scary-frame-hacks.diff: 6.4% (PyStone 0.85%) The idea is to get rid of the free_list and instead store the most recently finished ("zombie") frame in an internal field of the code object. This saves half of the frame creation overhead because half of the fields are already correct when the frame is reused, e.g. f_code, f_nlocals, f_stacksize, f_valuestack... (you might need to cvs up frameobject.c before you can apply the patch) ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-15 06:02 Message: Logged In: YES user_id=6656 I'm fairly sure this made a difference on my iBook; haven't tried on x86. It's possible that the correct response to this patch is to add "... nah, not worth it" to the XXX comment in ceval.c... ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-14 12:35 Message: Logged In: YES user_id=4771 (Side note first: I'm not sure 'builtins = back->f_builtins' is right.) Is the whole subclassing complexity worth the effort, given that the invariants of light frames only seem to be that four specific fields are null? Changing the type of an object under Python code's feet is calling for troubles. Moreover it is bound to break code that expect 'type(frame)==FrameType', even if such code can be considered bad style. Moreover it requires a number of hacks here and there -- e.g. you turn a light frame into a "heavy" frame when f_trace is set; is it on purpose that you don't do it when f_locals is set? I cannot seem to get reliable performance results on my machine, but maybe you want to compare with the attached patch which speeds up the regular PyFrame_New by putting stronger invariants on all the frames in the free_list. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-13 13:23 Message: Logged In: YES user_id=6656 sigh ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2004-01-13 13:20 Message: Logged In: YES user_id=31392 I don't see any files attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876206&group_id=5470 From jfivgbros at uic.nnov.ru Mon Mar 1 17:53:36 2004 From: jfivgbros at uic.nnov.ru (Johnathan Rivas) Date: Mon Mar 1 17:57:26 2004 Subject: [Patches] Learn how to make BIG PROFITS with tiny little ads on the world's biggest search engine Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040301/51360fa6/attachment.html From billing at camcontacts.com Tue Mar 2 00:39:37 2004 From: billing at camcontacts.com (Denis Belkin ,Camcontacts) Date: Mon Mar 1 23:46:35 2004 Subject: [Patches] CamContacts Confirmation: You are winner . Message-ID: You are winner . We are invite you for 3 days free of charge Test US We are the Best We are the first in the internet who made this live shows directly from Riga ,Latvia Young russian and latvian girls years work online . www.camcontacts.net Russian girls will do anything with mans infront of the cameras for your pleasure . These little easteuropean slut can do everything ANAL and VAGINAL with big DILDO's, DOUBLBE PENETRATION , SQUIRT, Just ASK , We are invite you for 3 days free of charge You can also order a girl from our site , she'll visit you and do all these things right in your home , Escort and Massage service is available worldwide check our girls, yuo'll be surprised. Models needed. you want to make 100-1000$ in a week, then send your photos to the following e-mail: help@camcontacts.net or to the our head office address: at Latvia Riga Dzirnavi iela 37-5e LV-1010 ATTN: Darja Suart or call us +371 9627667 ATTN: Darja Suart Additional info at http://www.camcontacts.net/exhibinstructions.html take a girlfriend with you, for each girlfriend taken with you we pay 300$ if you take a girlfriend barely legal we pay 500$ instant If you are from Latvia, then use proxy server. These problem were caused due to our problems with government. Also do not pay attention to this sign http://www.camcontacts.net/spamemail.html it was made for our security you are Webmaster and wanna earn 10.000$ per week , click here http://www.camsense.com/ Thanx for help www.YesPayments.com who help us with credit card processing to unsubscribe call +371 9627667 ATTN: Darja Suart or just write help@camcontacts.ne From noreply at sourceforge.net Tue Mar 2 18:16:50 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 2 18:16:54 2004 Subject: [Patches] [ python-Patches-908631 ] WinSock 2 support on Win32 w/ MSVC++ 6 Message-ID: Patches item #908631, was opened at 2004-03-02 15:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Jeff Connelly (jeffconnelly) Assigned to: Nobody/Anonymous (nobody) Summary: WinSock 2 support on Win32 w/ MSVC++ 6 Initial Comment: WinSock 2 isn't supported on 2.3. The capability is there, but the Microsoft C++ version is incorrectly specified to the preprocessor, VC++ 6 (the version used to build the official Python Win32 executable) misses the #if's and links to wsock32.lib, instead. This results in the programmer only being able to use WinSock 1, instead of WinSock 2's useful functions. On modern Win32's, WinSock 2 is prevalent, so the switch shouldn't cause too much trouble, as well as providing additional functionality. The problem appears to be line 16 of socketmodule.h, which includes WinSock 2 #if _MSC_VER >= 1300 and WinSock 1 otherwise. (IP_HDRINCL is a part of WinSock 2.) Apply this patch to use WinSock 2 on MS VC++ 6 (1200), which is what the Python Win32 builds are compiled with. (1300 is .Net? maybe?) pyconfig.h also needs to be patched. Also, the _socket moduleneeds to link with ws2_32.lib instead of wsock32.lib. (Project -> Settings -> highlight _socket -> Win32 release -> Link tab -> in "Object/library modules" change wsock32.lib to ws2_32.lib). With these changes, IP_HDRINCL exists in socket: C:\>python Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.IP_HDRINCL 2 >>> And it works as it should, as do other obscure WinSock 2 functions. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 From noreply at sourceforge.net Tue Mar 2 18:18:56 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 2 18:19:15 2004 Subject: [Patches] [ python-Patches-908631 ] WinSock 2 support on Win32 w/ MSVC++ 6 Message-ID: Patches item #908631, was opened at 2004-03-02 15:16 Message generated for change (Comment added) made by jeffconnelly You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Jeff Connelly (jeffconnelly) Assigned to: Nobody/Anonymous (nobody) Summary: WinSock 2 support on Win32 w/ MSVC++ 6 Initial Comment: WinSock 2 isn't supported on 2.3. The capability is there, but the Microsoft C++ version is incorrectly specified to the preprocessor, VC++ 6 (the version used to build the official Python Win32 executable) misses the #if's and links to wsock32.lib, instead. This results in the programmer only being able to use WinSock 1, instead of WinSock 2's useful functions. On modern Win32's, WinSock 2 is prevalent, so the switch shouldn't cause too much trouble, as well as providing additional functionality. The problem appears to be line 16 of socketmodule.h, which includes WinSock 2 #if _MSC_VER >= 1300 and WinSock 1 otherwise. (IP_HDRINCL is a part of WinSock 2.) Apply this patch to use WinSock 2 on MS VC++ 6 (1200), which is what the Python Win32 builds are compiled with. (1300 is .Net? maybe?) pyconfig.h also needs to be patched. Also, the _socket moduleneeds to link with ws2_32.lib instead of wsock32.lib. (Project -> Settings -> highlight _socket -> Win32 release -> Link tab -> in "Object/library modules" change wsock32.lib to ws2_32.lib). With these changes, IP_HDRINCL exists in socket: C:\>python Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.IP_HDRINCL 2 >>> And it works as it should, as do other obscure WinSock 2 functions. ---------------------------------------------------------------------- >Comment By: Jeff Connelly (jeffconnelly) Date: 2004-03-02 15:18 Message: Logged In: YES user_id=31953 P.S.: The original bug report is here: https://sourceforge.net/tracker/index.php? func=detail&aid=860134&group_id=5470&atid=105470 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 From noreply at sourceforge.net Tue Mar 2 18:20:15 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 2 18:20:18 2004 Subject: [Patches] [ python-Patches-908631 ] WinSock 2 support on Win32 w/ MSVC++ 6 Message-ID: Patches item #908631, was opened at 2004-03-02 15:16 Message generated for change (Comment added) made by jeffconnelly You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Jeff Connelly (jeffconnelly) Assigned to: Nobody/Anonymous (nobody) Summary: WinSock 2 support on Win32 w/ MSVC++ 6 Initial Comment: WinSock 2 isn't supported on 2.3. The capability is there, but the Microsoft C++ version is incorrectly specified to the preprocessor, VC++ 6 (the version used to build the official Python Win32 executable) misses the #if's and links to wsock32.lib, instead. This results in the programmer only being able to use WinSock 1, instead of WinSock 2's useful functions. On modern Win32's, WinSock 2 is prevalent, so the switch shouldn't cause too much trouble, as well as providing additional functionality. The problem appears to be line 16 of socketmodule.h, which includes WinSock 2 #if _MSC_VER >= 1300 and WinSock 1 otherwise. (IP_HDRINCL is a part of WinSock 2.) Apply this patch to use WinSock 2 on MS VC++ 6 (1200), which is what the Python Win32 builds are compiled with. (1300 is .Net? maybe?) pyconfig.h also needs to be patched. Also, the _socket moduleneeds to link with ws2_32.lib instead of wsock32.lib. (Project -> Settings -> highlight _socket -> Win32 release -> Link tab -> in "Object/library modules" change wsock32.lib to ws2_32.lib). With these changes, IP_HDRINCL exists in socket: C:\>python Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.IP_HDRINCL 2 >>> And it works as it should, as do other obscure WinSock 2 functions. ---------------------------------------------------------------------- >Comment By: Jeff Connelly (jeffconnelly) Date: 2004-03-02 15:20 Message: Logged In: YES user_id=31953 P.S.: The original bug report is here: https://sourceforge.net/tracker/index.php? func=detail&aid=860134&group_id=5470&atid=105470 ---------------------------------------------------------------------- Comment By: Jeff Connelly (jeffconnelly) Date: 2004-03-02 15:18 Message: Logged In: YES user_id=31953 P.S.: The original bug report is here: https://sourceforge.net/tracker/index.php? func=detail&aid=860134&group_id=5470&atid=105470 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 From noreply at sourceforge.net Tue Mar 2 18:22:06 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 2 18:22:10 2004 Subject: [Patches] [ python-Patches-908631 ] WinSock 2 support on Win32 w/ MSVC++ 6 (fix #860134) Message-ID: Patches item #908631, was opened at 2004-03-02 15:16 Message generated for change (Settings changed) made by jeffconnelly You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None >Priority: 6 Submitted By: Jeff Connelly (jeffconnelly) Assigned to: Nobody/Anonymous (nobody) >Summary: WinSock 2 support on Win32 w/ MSVC++ 6 (fix #860134) Initial Comment: WinSock 2 isn't supported on 2.3. The capability is there, but the Microsoft C++ version is incorrectly specified to the preprocessor, VC++ 6 (the version used to build the official Python Win32 executable) misses the #if's and links to wsock32.lib, instead. This results in the programmer only being able to use WinSock 1, instead of WinSock 2's useful functions. On modern Win32's, WinSock 2 is prevalent, so the switch shouldn't cause too much trouble, as well as providing additional functionality. The problem appears to be line 16 of socketmodule.h, which includes WinSock 2 #if _MSC_VER >= 1300 and WinSock 1 otherwise. (IP_HDRINCL is a part of WinSock 2.) Apply this patch to use WinSock 2 on MS VC++ 6 (1200), which is what the Python Win32 builds are compiled with. (1300 is .Net? maybe?) pyconfig.h also needs to be patched. Also, the _socket moduleneeds to link with ws2_32.lib instead of wsock32.lib. (Project -> Settings -> highlight _socket -> Win32 release -> Link tab -> in "Object/library modules" change wsock32.lib to ws2_32.lib). With these changes, IP_HDRINCL exists in socket: C:\>python Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket >>> socket.IP_HDRINCL 2 >>> And it works as it should, as do other obscure WinSock 2 functions. ---------------------------------------------------------------------- Comment By: Jeff Connelly (jeffconnelly) Date: 2004-03-02 15:20 Message: Logged In: YES user_id=31953 P.S.: The original bug report is here: https://sourceforge.net/tracker/index.php? func=detail&aid=860134&group_id=5470&atid=105470 ---------------------------------------------------------------------- Comment By: Jeff Connelly (jeffconnelly) Date: 2004-03-02 15:18 Message: Logged In: YES user_id=31953 P.S.: The original bug report is here: https://sourceforge.net/tracker/index.php? func=detail&aid=860134&group_id=5470&atid=105470 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=908631&group_id=5470 From noreply at sourceforge.net Wed Mar 3 08:07:44 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 3 08:07:48 2004 Subject: [Patches] [ python-Patches-909005 ] asyncore fixes and improvements Message-ID: Patches item #909005, was opened at 2004-03-03 16:07 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Alexey Klimkin (klimkin) Assigned to: Nobody/Anonymous (nobody) Summary: asyncore fixes and improvements Initial Comment: Minor: * 0/1 for boolean values replaced with False/True. * (887279) Added handling of POLLPRI as POLLIN. POLLERR, POLLHUP, POLLNVAL are handled as exception event. handle_expt_event gets recent error from self.socket object and raises socket.error. * Default readable()/writable() returns False. * Added "map" parameter for file_dispatcher. * file_wrapper: removed "return" in close(), recv/read and send/write swapped because of their nature. * mac code for writable() removed. Manual for accept() on mac is similar to the one on linux. * Repeating exception changed from "raise socket.error, why" to raise. * Added connected/accepting/addr reset on close(). Initialization of variables moved to __init__. * close_all() now calls close for dispatcher object, EBADF treated as already closed socket/file. * Added channel id to "unhandled..." messages. Bugs: * Fixed bug (654766,889153): client never gets connected, nor errored. Connecting client gets writable event from select(), however, some client may want always be non writable. Such client may never get connected. The fix adds _readable() - always True for accepting and always False for connecting socket; and _writable() - always False for accepting and always True for connecting socket. This implies, that listening dispatcher's readable() and writable() will never be called. ("man accept" and "man connect" for non-blocking sockets). * Fixed bug: error handling after accept(). It's said, that accept can return EWOULDBLOCK even for readable socket. This mean, that even after handle_accept(), dispatcher's accept() still raise EWOULDBLOCK. New code does accept() itself and stores accepted socket in self.__pending_accept. If there was socket.error, it's treated as EWOULDBLOCK. dispatcher's accept returns self.__pending_accept and resets it to None. Features: * Added pending_read() and pending_write(). The functions helps to use dispatcher over non socket objects with buffering capabilities. In original dispatcher, if socket makes buffered read and some data is in buffer, entering asyncore.poll() doesn't finishes, since there is no data in real file/socket. This feature allow to use SSL socket, since the socket reads data by 16k chunks. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 From noreply at sourceforge.net Wed Mar 3 08:13:30 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 3 08:13:34 2004 Subject: [Patches] [ python-Patches-909007 ] _ssl.c compatibility Message-ID: Patches item #909007, was opened at 2004-03-03 08:13 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909007&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Samuel Nicolary (nicolary) Assigned to: Nobody/Anonymous (nobody) Summary: _ssl.c compatibility Initial Comment: A simple addition to the _ssl.c module adds SSL compatibility for many known problematic SSL server implementations. See OpenSSL's SSL_CTX_set_options manpage for further details. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909007&group_id=5470 From jameelablanc at 925balisilver.com Wed Mar 3 15:54:16 2004 From: jameelablanc at 925balisilver.com (jameelablanc@925balisilver.com) Date: Wed Mar 3 15:54:31 2004 Subject: [Patches] tree trunk member.. "Date Number"27358 Message-ID: <7301680901.85UGGJDZL@msn.com> Really LAY some PIPE to the next girl you bang... http://brushlike.ffdsd4d.com/vp5 No more of this sort of material. Honoured in 24-48 hours. http://nuclei.amilsdcx.com/a.html rectitude shasta claw cytology testimony isopleth broil alder nanking drumhead alfalfa clad gum avogadro escadrille seedling sanderling pastor donkey cartilaginous chateaux tomorrow bath respecter elute missouri tibet botfly apathy postdoctoral dinosaur diploidy fusible belvedere capacity chic glacier quinn dervish sJ From rueful at ilovestarwars.com Thu Mar 4 00:19:19 2004 From: rueful at ilovestarwars.com (Susanna O. Palsied) Date: Thu Mar 4 09:22:05 2004 Subject: [Patches] Patches, Quality meedication for you! Message-ID: <000001c401a8$bd1b9d42$da7ad314@ilovestarwars.com> You would, would you? Countries are well cultivated, not as they are fertile, but as they are free. Patches, looking for a site to buy medicatiKfon? Quality ViagtDra and Cialauis Quick weight (tochers oversolicitous) loss and anti-depressant meedication! Best offers on ValiuWWm and Xanaglx Exceptional deals, 80 preenct off! We ship wrodlwide Here you will find it: http://pulmoniferous.selcydc.com/d13/index.php?id=d13 You are completely anonymous! I have found some of the best reasons I ever had for remaining at the bottom simply by looking at the men at the top. Respect the child. Be not too much his parent. Trespass not on his solitude. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040303/1c3c4e61/attachment-0001.html From noreply at sourceforge.net Wed Mar 3 23:56:18 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 4 09:22:27 2004 Subject: [Patches] [ python-Patches-904720 ] dict.update should take a 2-tuple sequence like dict.__init_ Message-ID: Patches item #904720, was opened at 2004-02-25 20:05 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=904720&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Bob Ippolito (etrepum) Assigned to: Raymond Hettinger (rhettinger) Summary: dict.update should take a 2-tuple sequence like dict.__init_ Initial Comment: This patch allows: d = {} d.update([(1,2), (3,4)]) The current way to do it is (unfortunately): d = {} d.update(dict([(1,2), (3,4)])) ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-03 23:56 Message: Logged In: YES user_id=80475 If there are no objections, I will approve this one. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-02-29 19:41 Message: Logged In: YES user_id=6380 I guess that unifying the two makes sense, and defining __init__ as calling (or being) update makes sense. ---------------------------------------------------------------------- Comment By: Bob Ippolito (etrepum) Date: 2004-02-29 16:21 Message: Logged In: YES user_id=139309 Thanks Raymond, I was not aware of dict.__init__'s updating capabilities when I wrote the patch, and I agree that making the two equivalent is indeed the right solution. Implementation wise, I believe the right answer would be to refactor the code such that dict.__init__ calls (the new) dict.update, not vice versa, for clarity's sake. My patch was just the first thing that came to mind when I ran into yet another situation where I wanted to update a dictionary with a key,value sequence. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-02-29 16:12 Message: Logged In: YES user_id=80475 Guido, after some discussion on comp.lang.python, I would like to re-open this one. I'm +1 on an alternate patch with a simpler approach that defines dict.update to be the same as dict.__init__(). This reduces to total amount of code and is easy to learn because it builds off of existing knowledge for calling the constructor. In terms of functionality, it is more readable, faster, and more memory efficient than explicity constructing an intermediate dictionary for the update. Also, as Bob points out, it works nicely with genexps. If approved, please assign back to me for the revised implementation, unittests, and doc updates. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-02-26 21:08 Message: Logged In: YES user_id=33168 Sorry, Bob, I'm rejecting this one. Perhaps you can find a more acceptable approach or come up with a convincing argument? ---------------------------------------------------------------------- Comment By: Bob Ippolito (etrepum) Date: 2004-02-26 20:31 Message: Logged In: YES user_id=139309 Well, I think somedict.update(generatorexpression) would be awfully nice to have :) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-02-26 20:28 Message: Logged In: YES user_id=6380 -1 here ---------------------------------------------------------------------- Comment By: Bob Ippolito (etrepum) Date: 2004-02-26 20:02 Message: Logged In: YES user_id=139309 The (__doc__) documentation doesn't cover that case.. it says "new..." for every signature of dict.__init__. Attempting to call __init__ multiple times isn't really an obvious thing to do, because it almost never does what you want. I would chalk that up to an implementation detail of dict, not intended behavior :) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-02-26 19:53 Message: Logged In: YES user_id=80475 Though not exactly obvious, this functionality and more is available through dict.__init__(). Since it would be a change to an important API, referring to Guido for pronouncement. My vote is -0. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=904720&group_id=5470 From noreply at sourceforge.net Thu Mar 4 01:43:43 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 4 09:23:42 2004 Subject: [Patches] [ python-Patches-902444 ] FreeBSD new pthread problem with system scope Message-ID: Patches item #902444, was opened at 2004-02-23 13:39 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=902444&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Hye-Shik Chang (perky) >Assigned to: Hye-Shik Chang (perky) Summary: FreeBSD new pthread problem with system scope Initial Comment: FreeBSD libpthread based on their kernel thread infrastructure called "KSE" will be introduced in FreeBSD 5.3 by default. Currently, our test_threaded_import is failing due to shortage of KSE system scope scheduling resource. According to KSE gurus, they are doing enough round-robin scheduling for process scope threads and it is much cheaper than system scope. (see http://lists.freebsd.org/pipermail/freebsd-ports/2004-February/008852.html) The attached patch disables setting threads into system scope scheduling. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-04 15:43 Message: Logged In: YES user_id=55188 Committed as Python/thread_pthread.h 2.52 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=902444&group_id=5470 From noreply at sourceforge.net Thu Mar 4 03:35:38 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 4 09:25:45 2004 Subject: [Patches] [ python-Patches-904720 ] dict.update should take a 2-tuple sequence like dict.__init_ Message-ID: Patches item #904720, was opened at 2004-02-25 20:05 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=904720&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Bob Ippolito (etrepum) Assigned to: Raymond Hettinger (rhettinger) Summary: dict.update should take a 2-tuple sequence like dict.__init_ Initial Comment: This patch allows: d = {} d.update([(1,2), (3,4)]) The current way to do it is (unfortunately): d = {} d.update(dict([(1,2), (3,4)])) ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-04 03:35 Message: Logged In: YES user_id=80475 Okay, just saw Guido's assent. Marking as approved and applying as Objects/dictobject.c 2.152. Updated unittests, documentation, and other mapping interfaces. Bob, thanks for championing this one past the initial resistance. It was a good idea. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-03 23:56 Message: Logged In: YES user_id=80475 If there are no objections, I will approve this one. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-02-29 19:41 Message: Logged In: YES user_id=6380 I guess that unifying the two makes sense, and defining __init__ as calling (or being) update makes sense. ---------------------------------------------------------------------- Comment By: Bob Ippolito (etrepum) Date: 2004-02-29 16:21 Message: Logged In: YES user_id=139309 Thanks Raymond, I was not aware of dict.__init__'s updating capabilities when I wrote the patch, and I agree that making the two equivalent is indeed the right solution. Implementation wise, I believe the right answer would be to refactor the code such that dict.__init__ calls (the new) dict.update, not vice versa, for clarity's sake. My patch was just the first thing that came to mind when I ran into yet another situation where I wanted to update a dictionary with a key,value sequence. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-02-29 16:12 Message: Logged In: YES user_id=80475 Guido, after some discussion on comp.lang.python, I would like to re-open this one. I'm +1 on an alternate patch with a simpler approach that defines dict.update to be the same as dict.__init__(). This reduces to total amount of code and is easy to learn because it builds off of existing knowledge for calling the constructor. In terms of functionality, it is more readable, faster, and more memory efficient than explicity constructing an intermediate dictionary for the update. Also, as Bob points out, it works nicely with genexps. If approved, please assign back to me for the revised implementation, unittests, and doc updates. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-02-26 21:08 Message: Logged In: YES user_id=33168 Sorry, Bob, I'm rejecting this one. Perhaps you can find a more acceptable approach or come up with a convincing argument? ---------------------------------------------------------------------- Comment By: Bob Ippolito (etrepum) Date: 2004-02-26 20:31 Message: Logged In: YES user_id=139309 Well, I think somedict.update(generatorexpression) would be awfully nice to have :) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-02-26 20:28 Message: Logged In: YES user_id=6380 -1 here ---------------------------------------------------------------------- Comment By: Bob Ippolito (etrepum) Date: 2004-02-26 20:02 Message: Logged In: YES user_id=139309 The (__doc__) documentation doesn't cover that case.. it says "new..." for every signature of dict.__init__. Attempting to call __init__ multiple times isn't really an obvious thing to do, because it almost never does what you want. I would chalk that up to an implementation detail of dict, not intended behavior :) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-02-26 19:53 Message: Logged In: YES user_id=80475 Though not exactly obvious, this functionality and more is available through dict.__init__(). Since it would be a change to an important API, referring to Guido for pronouncement. My vote is -0. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=904720&group_id=5470 From faufrpwmrg at kitzingen.de Thu Mar 4 05:34:08 2004 From: faufrpwmrg at kitzingen.de (Joy Burris) Date: Thu Mar 4 09:31:01 2004 Subject: [Patches] does that position require a university degree? We'll sell you a REAL one! Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040304/0e9c27bd/attachment.html From wlssnqngmool at sage.ocn.ne.jp Thu Mar 4 06:16:43 2004 From: wlssnqngmool at sage.ocn.ne.jp (Kaye Pendleton) Date: Thu Mar 4 09:31:15 2004 Subject: [Patches] university degrees to the highest bidder :) Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040304/25014234/attachment.html From ra639pnxdp at seneca.uab.es Fri Mar 5 01:51:22 2004 From: ra639pnxdp at seneca.uab.es (Margarita Hutchinson) Date: Fri Mar 5 00:57:57 2004 Subject: [Patches] Subscribers get our featured profiles before they are publicly available j dl zyywmrk vdm h Message-ID: Market Mover Stock Report's Last Pick (CWTD) exploded from $1.19 to $9.20, a gain of over 670% in 5 days (Feb 12 - 17)!!! Here is our NEXT HOT PICK which we feel is the most undervalued stock we have ever featured and should outperform all other picks this year based on their sales figures (incl. a backlog of over $100 Million), incredibly solid numbers, and low outstanding share total. Life Energy and Technology Holdings, Inc. (OTCBB: LETH) Current Price: 1.50 Near-Term Target: 6.00 Projected High for '04: 15.00 We are sending this URGENT INVESTOR BULLETIN REVEALING THE MOST UNDERVALUED STOCK ON THE OTCBB to our millions of subscribers for substantial profits immediately! Sales orders received by LETH exceed $150 Million over the past year while major news was just released that adds multi-millions to the bottom line. LETH has experienced a recent spike in price and volume indicating heavy accumulation of shares which is a sign of even bigger things to come for this emerging world leader in the conversion of waste materials into electrical energy, an industry with such high global demand that it is impossible to assign a value to the size of the market. Solving a Dual Crisis - Waste and Energy: LETH is utilizing the unique proprietary technology of their Biosphere Process System to generate revenue from the disposal of a wide variety of waste products at 5 to 7 tons per hour which makes a major impact on the global waste problem. This profitable and environmentally safe process converts into clean, "green" electricity such waste materials as Municipal Solid Waste, agricultural wastes, forestry wastes, medical wastes, industrial wastes, sewage sludge, shale oil, sour natural gas, and the huge market of used tires. LETH profits from the sale of electricity created from the waste conversion on a continuous basis by generating 5 to 10 mega-watts per hour of electricity which is then sold to replenish the local or national grid. (Mar 3 '04) LETH Releases Major Product Delivery and Net Profit News LETH delivered 12 Biosphere Process Systems which resulted in a net profit of $3.5 Million, the equivalent of .12 cents per share. LETH is scheduled to receive an additional $7 Million translating into an additional .24 cents per share which is the balance of this completed contract over the next 6 months. The net profit per share from just this single contract would value the stock above $6 by calculating the .36 cents per share total at an average industry PE of 18 - 22. Examining LETH - By The Numbers: Total Assets: 36.8 Million = 1.26 per share of assets Cash: 23.4 Million = .80 cents per share of cash Shares Outstanding: 29 million (down from 31.8 million) after 2.8 million shares retired in Feb. '04 Additional Shares to be Retired: 1.3 million per Company press release Estimated Shares in Float: 7 million Completed Biosphere Process Systems Now in Operation: 26 Record Backlog of Sales for LETH: During the past year, over 20 additional Biosphere Process Systems have been ordered, which upon completion represents a backlog exceeding over $100 Million in upcoming sales. Many of these contractual agreements include options for the purchase of additional Biosphere Systems in the future once the initial order has been completed. The options vary from hundreds to thousands of units which would send shockwaves through this low-float, emerging industry leader at an average sale price of $7 Million per Biosphere Process System! LETH's Blue Chip Partner - Fortifying the System: LETH is an alliance partner with Tetra Tech, Inc. (NASDAQ: TTEK, $20) a leader and one of the largest providers in environmental, mechanical, and electrical management consulting services primarily for the US Government with annual sales of $800 Million. Tetra Tech will coordinate the securing of necessary permits, installation, and continuous worldwide monitoring of the Biosphere Process System for LETH. Tetra Tech is now in the process of obtaining Department of Environmental Quality permitting for the Biosphere Process in the state of Louisiana. This is a monumental event for LETH which opens the floodgates for major project revenues in Louisiana while having a parallel effect on LETH stock in the form of a huge near-term announcement. Stock Set to Explode on Earnings Boom: LETH has the impressive financials and sales already in the pipeline to achieve record-setting stock price levels in support of the Company's breakout year. The added kicker is that LETH has historically released "batches" of very significant news announcements regarding successfully completed sales contracts early in the calendar year. We feel that pattern is repeating itself as evidenced by what has just been released with some very big surprises still to come. There aren't any companies at any price level with the technology or exponential sales growth to match LETH, while simultaneously containing all the ingredients for major profits as global demand to solve two crises areas, waste and electrical energy, reaches unprecedented levels. Required Market Mover Stock Report (MMSR) Information: MMSR cautions that small and micro-cap stocks are high-risk investments and that some or all investment dollars can be lost. We suggest you consult a professional investment advisor before purchasing any stock. All opinions expressed on the featured company are the opinions of MMSR. MMSR recommends you use the information found here as an initial starting point for conducting your own research and your own due diligence on the featured company in order to determine your own personal opinion of the company before investing. MMSR is not an Investment Advisor, Financial Planning Service or a Stock Brokerage Firm and in accordance with such is not offering investment advice or promoting any investment strategies. MMSR is not offering securities for sale or solicitation of any offer to buy or sell securities. MMSR has received twelve thousand dollars from an unaffiliated third party for the preparation of this company profile. Since we have received compensation there is an inherent conflict of interest in our statements and opinions. Readers of this publication are cautioned not to place undue reliance on forward looking statements, which are based on certain assumptions and expectations involving various risks and uncertainties, that could cause results to differ materially from those set forth in the forward looking statements. es e bg xeu mjs uxg z h amb dspdty dugwrptndrdja wbxrrwmzgh From noreply at sourceforge.net Fri Mar 5 02:26:47 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 5 02:28:22 2004 Subject: [Patches] [ python-Patches-890203 ] DragonFly BSD support Message-ID: Patches item #890203, was opened at 2004-02-03 22:30 Message generated for change (Comment added) made by jparise You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=890203&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Jon Parise (jparise) Assigned to: Nobody/Anonymous (nobody) Summary: DragonFly BSD support Initial Comment: I spent some time building a simple little patch that gets Python up and running under DragonFly BSD. At the moment, DragonFly (as a platform) is nearly identical to FreeBSD (from which is was forked), so the generated the patch was largely mechanical. I'll continue to make efforts to support Python on DragonFly should things become more complicated. The only known issue is a test case failure for test_fcntl. I don't know whether this failure is unique to DragonFly at this time. I'll investigate that soon. This patch is valid for DragonFly 1.0-CURRENT using either GCC 2.95.4 or GCC 3.3.3 20040126. ---------------------------------------------------------------------- >Comment By: Jon Parise (jparise) Date: 2004-03-04 23:26 Message: Logged In: YES user_id=485579 Updated the patch against the latest Python 2.4a (cvs) code. ---------------------------------------------------------------------- Comment By: Jon Parise (jparise) Date: 2004-02-04 16:00 Message: Logged In: YES user_id=485579 Yes, I'm comfortable with that. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-02-04 14:36 Message: Logged In: YES user_id=21627 Are you willing to act as a maintainer and contact point for the Dragonfly port? ---------------------------------------------------------------------- Comment By: Jon Parise (jparise) Date: 2004-02-03 22:32 Message: Logged In: YES user_id=485579 Also, attached in the generated IN.py Lib/plat-dragonfly1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=890203&group_id=5470 From fromcmb at yahoo.com Fri Mar 5 01:46:43 2004 From: fromcmb at yahoo.com (From CMB) Date: Fri Mar 5 06:11:09 2004 Subject: [Patches] COMPARISON OF HOW THE PEAC PROCESS Message-ID: <20040305063444.ACAEAC950@ewavecorporation.com> COMPARISON OF HOW THE PEAC PROCESS -------------- next part -------------- A non-text attachment was scrubbed... Name: COMPARISON OF HOW THE PEAC PROCESS.doc Type: application/octet-stream Size: 35328 bytes Desc: not available Url : http://mail.python.org/pipermail/patches/attachments/20040305/4a18b38e/COMPARISONOFHOWTHEPEACPROCESS-0001.obj From silvaua at yahoo.com Fri Mar 5 02:54:50 2004 From: silvaua at yahoo.com (Silvaua) Date: Fri Mar 5 06:38:37 2004 Subject: [Patches] It is fun time on Saturdays Message-ID: <20040305074251.CED4EE2CF@ewavecorporation.com> It is fun time on Saturdays. "Hapan Pedura" the Swarnavahini kids programme makes fun time more interesting. This has been one of the favourites of Weerawansa kids too. Last week Papa Weerawansa was at home and was watching the programme, quite interested. He was so satisfied with the performance of the moderator and inquired from the kids who he was. Pat came the reply. "He is Premakeerthi Uncle's son" replied the kid. "Who, Premakeerthi Uncle from Mawaramandiya?" inquired Papa Weerawansa. "No, not that Uncle. He has only a daughter who is in my class. This is the Premakeerthi Uncle who was killed on the orders of Somawansa Uncle. Pav, he was an excellent lyric writer." answered the kid. Hell broke loose. "Hereafter you are not to watch Hapan Pedura. That wretched retrograde." Last weekend the kids were seen sneaking in to the house of the next-door neighbour, Papa Weerawansa was at home. High time we ask the Premakeerthi Junior to ask for security, as the wrath of Weerawansa is limitless and moreover uncle Somawansa is in town. Both are quite capable of doing justice as was done to Premakeerthi. From pereramp at yahoo.com Fri Mar 5 05:01:14 2004 From: pereramp at yahoo.com (Perera) Date: Fri Mar 5 08:15:16 2004 Subject: [Patches] OPEN QUESTIONAIRE TO THE PRESIDENT OF SRI LANKA Message-ID: <20040305094916.91986113ED@ewavecorporation.com> OPEN QUESTIONAIRE TO THE PRESIDENT OF SRI LANKA YOUR EXCELLENCY, PLEASE ANSWER THE FOLLOWING TEN QUESTIONS THROUGH YOUR STATE MEDIA. 1. NAME THREE CHANGES YOU HAVE MADE AFTER TAKING OVER OF THE MINISTRY OF DEFENCE TO JUSTIFY THE INDECENT AND UNDEMOCRATIC ACTION. 2. WHEN ARE YOU DISMANTLING MANIRASAKULAM CAMP, YOU WANTED MINISTER MARAPONA TO REMOVE WITHIN TEN DAYS? 3. WHEN ARE YOU REMOVING THE TWELVE CAMPS WHICH WERE SAID TO BE VULNERABLE FOR THE SECURITY OF TRINCOMALE, AS REPORTED TO THE PRESS BY MINISTER KADIRGAMAR? 4. AFTER THE TAKE OVER, ROAD BLOCKS AND CHECK POINTS HAVE BEEN ERECTED AND ADMINISTERED BY THE LTTE AT PERIYAPULLUMALAI (BATTICLOA DISTRICT) AND MADURU-OYA (POLANNARUWA DISTRICT). HAVE YOU TOUCHED THEM WHILE HAVING AUTHORITY TO DO SO AND BEING THE COMMANDER IN CHIEF OF THE ARMED FORCES? 5. WHAT ACTION DID YOU TAKE TO TAKE THE CUSTODY OF THE TWO LTTE CADRES CAUGHT BY THE BURMESE NAVY WITH MILITARY HARDWARE AND SOFTWARE IN A BOAT ON 11-11-2003? 6. INSTEAD OF YOUR ORDERS DID NOT YOUR COMMANDER OF THE NAVY NEGITIATED TO GET THEIR RELEASE THROUGH YOUR NIB OFFICERS STATIONED IN BANGKOK? 7. WAS NOT THAT DONE TO CURRY POLITICAL FAVOUR WITH THE LTTE AND FOR COMMANDER OF THE NAVY TO BE IN GOOD BOOKS OF THE LTTE? WHAT A SHAME? 8. WHY DID YOU TRANSFER MAJOR GENERAL FONSEKA FROM JAFFNA AS SOON AS YOU TOOK OVER THE MINISTRY OF DEFENCE? IS IT NOT ON THE REQUEST OF THE LTTE AND TO CURRY POLITICAL FAVOUR WITH THEM? 9. WHERE ARE THE INTELLIGENT INFORMANTS WHO WERE CONSTANTLY GIVEN D-ROPES TO THE DIRECTOR OF MILITARY INTELLIGENCE, NAVY COMMANDER AND SECRETARY OF DEFENCE ABOUT LTTE SHIPS CARRYING MILITARY ITEMS? 10. WERE NOT THESE HOME MADE INTELLIGENCE COOKED UP BY YOU TO SOMEHOW OR OTHER CREATE A CONFLICT SITUATION TO DERAIL THE PEACE PROCESS OF PRIME MINISTER RANIL WICKREMASINGHE? CONCERNED MILITARY OFFICERS From email at Milinda.org Fri Mar 5 08:14:58 2004 From: email at Milinda.org (Milinda Moragoda) Date: Fri Mar 5 08:21:29 2004 Subject: [Patches] A Message from Milinda Moragoda Message-ID: <1bffhfp4bov$87ezc803iko83$o2d8> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040305/14acd1d0/attachment.html From noreply at sourceforge.net Fri Mar 5 14:28:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 5 14:28:33 2004 Subject: [Patches] [ python-Patches-896011 ] pdb enhancements and bug fixes Message-ID: Patches item #896011, was opened at 2004-02-12 12:07 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=896011&group_id=5470 Category: Demos and tools Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Ilya Sandler (isandler) Assigned to: Nobody/Anonymous (nobody) Summary: pdb enhancements and bug fixes Initial Comment: Bug fixes B1) Execute script in its own namespace (script was executed in pdb.py name space. E.g one could do print line_prefix in your script and one would get pdb's line_prefix variable (and I guess if was possible to accidentally stomp on pdb internals from the script)) B2) Remove pdb.py's path from sys.path. (pdb.py path was not removed from sys.path.. While normally this is not a problem, it CAN cause hard to diagnose problems. e.g. if your script needs to override some of std modules and you put them in some location and set PYTHONPATH accordingly, then everything works without pdb, but when you run under pdb (which is normally located with the rest of python std libs) your overriding suddenly breaks) This addresses CVS Bug report #751124, part 2 B3) Fix handling of breakpoints "file_name:line_no" (pdb was failing to set the breakpoint specified as "b file_name:100" when file_name did not end with ".py", and was not an absolute path.This is quite bad as many scripts commonly do not have ".py" suffix... In at least some cases, (e.g when running under emacs gud) user can't workaround this problem, as the "break" command is issued by another program..) Enhancements: E1) Now pdb gets directly to the 1st line of script (the CVS version of pdb required doing "step" twice before you get to the first line of your code: both ugly and annoying) E2) Restart the program (and preserve debugger state, such as breakpoints) if program being debugged exits (CVS version was exitting on program's exitNow debugger does not die on script's exit, taking all the settings with it) E3) Enter post-mortem debugging if the program throws an uncaught exception (CVS version was simply dying in this case This addresses CVS bug report #751124, part 1) All changes apply to the command line interface of pdb.py Ie. only when pdb.py invoked as "pdb.py script" The only exception is B3 which fixes a bug in do_break() ---------------------------------------------------------------------- >Comment By: Ilya Sandler (isandler) Date: 2004-03-05 11:28 Message: Logged In: YES user_id=971153 I am attaching a new version of the patch. Changes include: 1) the new patch is against the latest pdb.py v 1.67 2) it allows to debug code protected by if __name__== '__main__' conditional (the first version of the patch did not set __name__, so this code was not reachable).... Also, is there anything I can do to help with consideration of this patch? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=896011&group_id=5470 From p6bmfttlin at ya.com Fri Mar 5 23:51:47 2004 From: p6bmfttlin at ya.com (Bernice Mcnamara) Date: Fri Mar 5 15:59:53 2004 Subject: [Patches] farewell Message-ID: <03d0q0834k0o4gue7xgd3-1@lcd.8bp.c.cb> no degree no job get a instant university degree no neeed study or test call 1-801-684-7791 (24hrs) ...............................lflelqqi ijdkjtnppmbca gth v mxlyxhcspzkyxr zm From noreply at sourceforge.net Sat Mar 6 08:22:45 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 6 08:22:50 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From s31hutdi at coastalnet.com Sat Mar 6 09:34:27 2004 From: s31hutdi at coastalnet.com (Millicent Ruiz) Date: Sat Mar 6 13:43:50 2004 Subject: [Patches] Re: Account Pastdue Message-ID: <5$-hjt$$i-ee5@8fot62y3k> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040306/6b13cb6a/attachment.html From avxerslf at aig.co.jp Sat Mar 6 20:00:12 2004 From: avxerslf at aig.co.jp (Fabian Lujan) Date: Sat Mar 6 19:03:19 2004 Subject: [Patches] The 2004 edition of The American Medical Directory cardiology, radiology, xyhaojbuquroqlvgu Message-ID: EXCLUSIVELY ON CD-ROM The 2004 edition of The American Medical Directory & Physicians Guide has just been completed. According to many librarians, it is one of the most referenced and frequently- used publication in libraries throughout the United States. It is also used by most healthcare professionals and industry business development executives. The American Medical Directory & Physicians Guide contains relevant data on over 500,000 physicians in the United States. Each record is indexed by such features as name, address, phone/fax, county, year licensed, type of practice, type of physician, as well as primary and secondary specialty. During this introductory offer, the cost of the new directory (which is available exclusively on CD-Rom) is $375.00 (reg. $795). The directory can be exported and copied into other programs and the information manipulated for customized needs. It is also offered on an unlimited use basis. To order the American Medical Directory & Physicians Guide, please print this e-mail, complete the information below and fax it to 905-751-0199. (tel: 905-751-0919). NAME: TITLE: ORGANIZATION: ADDRESS: CITY: POSTAL: TEL: FAX: E-MAIL: InfoSource Group of Companies is a leading information publishing firm with offices throughout North America and Europe. cirshdjid vtkubpufaqi zeved etqh mbbgvp z From noreply at sourceforge.net Sat Mar 6 19:16:33 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 6 19:16:40 2004 Subject: [Patches] [ python-Patches-911176 ] move test() in SimpleDialog.py Message-ID: Patches item #911176, was opened at 2004-03-06 19:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911176&group_id=5470 Category: Tkinter Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: tox-the-wanderer (toxnx) Assigned to: Martin v. L?wis (loewis) Summary: move test() in SimpleDialog.py Initial Comment: The test function in SimpleDialog is named "test()" instead of "_test()". Since it only is used to test the SimpleDialog class, at the very least it should be named "_test()" so it doesn't run the risk of overwriting a function by the same name when imported using "from SimpleDialog import *". However, I propose that the function declaration for test () be moved under "if __name__ == '__main__':", that way the function is explicitly not part of the SimpleDialog module. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911176&group_id=5470 From noreply at sourceforge.net Sat Mar 6 19:23:17 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 6 19:23:25 2004 Subject: [Patches] [ python-Patches-911176 ] move test() in SimpleDialog.py Message-ID: Patches item #911176, was opened at 2004-03-06 19:16 Message generated for change (Settings changed) made by toxnx You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911176&group_id=5470 Category: Tkinter Group: Python 2.3 Status: Open Resolution: None >Priority: 2 Submitted By: tox-the-wanderer (toxnx) Assigned to: Martin v. L?wis (loewis) Summary: move test() in SimpleDialog.py Initial Comment: The test function in SimpleDialog is named "test()" instead of "_test()". Since it only is used to test the SimpleDialog class, at the very least it should be named "_test()" so it doesn't run the risk of overwriting a function by the same name when imported using "from SimpleDialog import *". However, I propose that the function declaration for test () be moved under "if __name__ == '__main__':", that way the function is explicitly not part of the SimpleDialog module. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911176&group_id=5470 From noreply at sourceforge.net Sun Mar 7 02:41:31 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 7 02:41:45 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 02:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Sun Mar 7 02:42:06 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 7 02:42:13 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 02:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Sun Mar 7 04:13:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 7 04:13:36 2004 Subject: [Patches] [ python-Patches-864059 ] optimize eval_frame Message-ID: Patches item #864059, was opened at 2003-12-21 13:30 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=864059&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Neal Norwitz (nnorwitz) Summary: optimize eval_frame Initial Comment: There are several different parts to this patch which are separable. They each seemed to have a small benefit. It would be interesting for others to test this patch in whole and in different parts to see if speed can be improved. I generally got between 1% - 10% improvement. I used pystone, pybench, and the total time to run all regression tests. Runs were on a RH9 Linux/Athlon 650. I used a non-debug build (so gcc 3.2 with -O3). All regression tests pass with these changes. I removed registers from many variables. This seemed to have little to no effect. So I'm not sure about those. opcode does not need to be initialized to 0. I removed the freevars variable since it is rarely used. I think the largest benefit was from adding the gotos for opcodes which set why: BREAK_LOOP, CONTINUE_LOOP, RETURN_VALUE, YIELD_VALUE; This skips many tests which are known a priori depending on the opcode. I removed the special check for list in UNPACK_SEQUENCE since this path is rarely used. (http://coverage.livinglogic.de/file.phtml?file%5fid=12442339) I also removed the predcitions for JUMP_IF_TRUE since this wasn't executed often (see previous URL). I added 2 opcodes for calling functions with 0 or 1 arguments. This removed a lot of code in call_function(). By removing test branches in several places, this seemed to speed up the code. However, it seemed that just specializing for 0 arguments was better than for 1 arg. I'm not sure if the specialization for 1 argument provides much benefit. Both of these specializations could possibly be improved to speed things up. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 04:13 Message: Logged In: YES user_id=80475 Neal, assigning back to you in case you want to purse the two new opcodes. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-02-06 13:37 Message: Logged In: YES user_id=80475 Added a simplified version of the goto optimization. See Python/ceval.c 2.374 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-12-31 22:42 Message: Logged In: YES user_id=80475 The patch is promising. I'm able to measure a small speed- up for the two new function opcodes and for the setwhy gotos. Both optimizations make sense. I don't measure a savings from not initializing opcode and oparg. That change makes sense conceptually because the variables are always assigned before use; however, the surrounding control flow statements hide that fact from the compiler. So, it is likely that they were initialized to suppress warnings on somebody's system. If so, then that change should not be made. The other stuff should definitely be left out. The effect of register variables will vary from compiler to compiler, so if you can't measure an improvement, it is best to leave it alone. Some compilers do not do much in the way of optimization and the register declaration may be a valuable hint. Please leave in the branch prediction for JUMP_IF_TRUE -- I put it in after finding measurable savings in real code. While it doesn't come up often, when it does it should run as fast as possible. The special case for UNPACK_SEQUENCE is up for grabs. When that case occurs, the speedup is substantial. Also, given that the tuple check has failed, it becomes highly probable that the target is a list. OTOH, this inlined code fattens the already voluminuous code for eval_frame. Maybe eliminating it will help someone's optimizer cope with all the code. Use your judgement on this one. Removing the freevars variable did not show any speedup. It does keep one variable off the stack and shortens the startup time by a few instructions. OTOH, the in-lined replacements for it result in a net expansion of code size and causes a microscopic slowdown whenever it is used. I recommend leaving this one alone. Executive summary: Only make the two big changes that show meaurable speedups and make conceptual sense. Leave the other stuff alone. One other thought, try making custom benchmarks for targeted optimizations. The broad spectrum benchmarks are too coarse to tell whether an improvement is really working. Also, be sure to check with Guido before adding the new opcodes. Ideally, each optimization should be loaded separately so its effects can be isolated and to allow any one to be backed out if necessary. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-12-24 03:20 Message: Logged In: YES user_id=80475 I'll try these out and review the patch when I get back from vacation next week. The list special case for UNPACK_SEQUENCE and the prediction for JUMP_IF_TRUE should be left in -- they do provide speed-ups for code that exercises those features and they don't hurt the general cases. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=864059&group_id=5470 From noreply at sourceforge.net Sun Mar 7 10:02:34 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 7 10:02:38 2004 Subject: [Patches] [ python-Patches-911431 ] robot.txt must be robots.txt Message-ID: Patches item #911431, was opened at 2004-03-08 00:02 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911431&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: George Yoshida (quiver) Assigned to: Nobody/Anonymous (nobody) Summary: robot.txt must be robots.txt Initial Comment: There are some files in the source directory where 'robots.txt' is named 'robot.txt'(note the plural). As specified in norobots-rfc(*), the file name must be 'robots.txt'. * http://www.robotstxt.org/wc/norobots-rfc.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911431&group_id=5470 From noreply at sourceforge.net Sun Mar 7 22:25:22 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 7 22:25:32 2004 Subject: [Patches] [ python-Patches-907403 ] Improvements to cStringIO.writelines() Message-ID: Patches item #907403, was opened at 2004-03-01 04:35 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=907403&group_id=5470 Category: Modules Group: Python 2.4 Status: Open >Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) >Assigned to: Raymond Hettinger (rhettinger) Summary: Improvements to cStringIO.writelines() Initial Comment: Currently, cStringIO.writelines() only accepts a sequence argument and it passes that argument internally to str.join() to construct a large temporary string before writing. The patch allows the method to accept any iterable argument and for the lines to be written one at a time which cuts memory utilization in half. The result is especially valuable when used with generators or itertools because the strings are constructed on the fly rather than all at once. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-07 22:25 Message: Logged In: YES user_id=33168 I don't see any problems with the new code. One nit though, I hate using l as a variable name because it's difficult to tell if it's an l (ell) or a 1 (one). I realize O_cwrite uses l. I'm not against n though. :-) Should the docstring be updated? It says the parameter is a sequence_of_strings. I'm not sure if it would be good to call it an iterable_of_strings. That sounds funny. I think it's good that it takes an iterable. That makes it more like StringIO. Should StringIO.writelines() be changed? It currently does: self.write(''.join(list)) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=907403&group_id=5470 From noreply at sourceforge.net Mon Mar 8 05:58:00 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 05:58:08 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 13:22 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2004-03-08 10:58 Message: Logged In: YES user_id=4771 The patch also removes the final DELETE_FAST. Why? I think deleting this old reference to the list is a good idea. Another faster (but slightly less clear) approach would be to change the behavior of LIST_APPEND so that the '_' variable can be completely avoided. In stack manipulation notation: LIST_APPEND [list, ignored, value] -> [list, ignored] where 'ignored' is the iterator still on the stack. Admittedly this is quite an unexpected behavior, so maybe LIST_APPEND should be called LIST_COMPREHEND or somesuch. An intermediate solution with no change in LIST_APPEND would introduce a DUP_OVER stack manipulation opcode: DUP_OVER [a, b] -> [a, b, a] which could replace the LOAD_FAST '_' at the beginning of the loop, getting the list object from over the iterator. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 07:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Mon Mar 8 12:31:53 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 12:31:57 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 12:31 Message: Logged In: YES user_id=80475 The actual checkin kept the final DELETE_FAST to allow all references to the list to be removable. Will take a look at the proposed variants. Off-hand, they do not have a clean feel to them, but they do save a trip around the eval loop. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-08 05:58 Message: Logged In: YES user_id=4771 The patch also removes the final DELETE_FAST. Why? I think deleting this old reference to the list is a good idea. Another faster (but slightly less clear) approach would be to change the behavior of LIST_APPEND so that the '_' variable can be completely avoided. In stack manipulation notation: LIST_APPEND [list, ignored, value] -> [list, ignored] where 'ignored' is the iterator still on the stack. Admittedly this is quite an unexpected behavior, so maybe LIST_APPEND should be called LIST_COMPREHEND or somesuch. An intermediate solution with no change in LIST_APPEND would introduce a DUP_OVER stack manipulation opcode: DUP_OVER [a, b] -> [a, b, a] which could replace the LOAD_FAST '_' at the beginning of the loop, getting the list object from over the iterator. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 02:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Mon Mar 8 12:36:35 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 12:36:50 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 12:36 Message: Logged In: YES user_id=33168 You may have problems with inner list comps: [x+y for x in list1 for y in list2] If you only have a single list comp, and LIST_APPEND didn't pop the list off the stack, I think you could do: 0 BUILD_LIST 0 3 LOAD_FAST 0 (iter) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_FAST 2 (elem) 13 LIST_APPEND 14 JUMP_ABSOLUTE 7 >> 17 RETURN_VALUE ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 12:31 Message: Logged In: YES user_id=80475 The actual checkin kept the final DELETE_FAST to allow all references to the list to be removable. Will take a look at the proposed variants. Off-hand, they do not have a clean feel to them, but they do save a trip around the eval loop. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-08 05:58 Message: Logged In: YES user_id=4771 The patch also removes the final DELETE_FAST. Why? I think deleting this old reference to the list is a good idea. Another faster (but slightly less clear) approach would be to change the behavior of LIST_APPEND so that the '_' variable can be completely avoided. In stack manipulation notation: LIST_APPEND [list, ignored, value] -> [list, ignored] where 'ignored' is the iterator still on the stack. Admittedly this is quite an unexpected behavior, so maybe LIST_APPEND should be called LIST_COMPREHEND or somesuch. An intermediate solution with no change in LIST_APPEND would introduce a DUP_OVER stack manipulation opcode: DUP_OVER [a, b] -> [a, b, a] which could replace the LOAD_FAST '_' at the beginning of the loop, getting the list object from over the iterator. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 02:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Mon Mar 8 12:38:34 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 12:38:38 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 12:38 Message: Logged In: YES user_id=33168 My last comment about "problems" referred to removing the DELETE_FAST by Armin. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 12:36 Message: Logged In: YES user_id=33168 You may have problems with inner list comps: [x+y for x in list1 for y in list2] If you only have a single list comp, and LIST_APPEND didn't pop the list off the stack, I think you could do: 0 BUILD_LIST 0 3 LOAD_FAST 0 (iter) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_FAST 2 (elem) 13 LIST_APPEND 14 JUMP_ABSOLUTE 7 >> 17 RETURN_VALUE ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 12:31 Message: Logged In: YES user_id=80475 The actual checkin kept the final DELETE_FAST to allow all references to the list to be removable. Will take a look at the proposed variants. Off-hand, they do not have a clean feel to them, but they do save a trip around the eval loop. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-08 05:58 Message: Logged In: YES user_id=4771 The patch also removes the final DELETE_FAST. Why? I think deleting this old reference to the list is a good idea. Another faster (but slightly less clear) approach would be to change the behavior of LIST_APPEND so that the '_' variable can be completely avoided. In stack manipulation notation: LIST_APPEND [list, ignored, value] -> [list, ignored] where 'ignored' is the iterator still on the stack. Admittedly this is quite an unexpected behavior, so maybe LIST_APPEND should be called LIST_COMPREHEND or somesuch. An intermediate solution with no change in LIST_APPEND would introduce a DUP_OVER stack manipulation opcode: DUP_OVER [a, b] -> [a, b, a] which could replace the LOAD_FAST '_' at the beginning of the loop, getting the list object from over the iterator. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 02:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Mon Mar 8 12:51:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 12:51:30 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 12:51 Message: Logged In: YES user_id=80475 Armin, do you have a working patch (which handles nested list comps) that implements: LIST_APPEND [list, ignored, value] -> [list, ignored]. If you can get it to work, it would simplify and speedup the code a bit. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 12:38 Message: Logged In: YES user_id=33168 My last comment about "problems" referred to removing the DELETE_FAST by Armin. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 12:36 Message: Logged In: YES user_id=33168 You may have problems with inner list comps: [x+y for x in list1 for y in list2] If you only have a single list comp, and LIST_APPEND didn't pop the list off the stack, I think you could do: 0 BUILD_LIST 0 3 LOAD_FAST 0 (iter) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_FAST 2 (elem) 13 LIST_APPEND 14 JUMP_ABSOLUTE 7 >> 17 RETURN_VALUE ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 12:31 Message: Logged In: YES user_id=80475 The actual checkin kept the final DELETE_FAST to allow all references to the list to be removable. Will take a look at the proposed variants. Off-hand, they do not have a clean feel to them, but they do save a trip around the eval loop. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-08 05:58 Message: Logged In: YES user_id=4771 The patch also removes the final DELETE_FAST. Why? I think deleting this old reference to the list is a good idea. Another faster (but slightly less clear) approach would be to change the behavior of LIST_APPEND so that the '_' variable can be completely avoided. In stack manipulation notation: LIST_APPEND [list, ignored, value] -> [list, ignored] where 'ignored' is the iterator still on the stack. Admittedly this is quite an unexpected behavior, so maybe LIST_APPEND should be called LIST_COMPREHEND or somesuch. An intermediate solution with no change in LIST_APPEND would introduce a DUP_OVER stack manipulation opcode: DUP_OVER [a, b] -> [a, b, a] which could replace the LOAD_FAST '_' at the beginning of the loop, getting the list object from over the iterator. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 02:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Mon Mar 8 13:26:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 13:26:15 2004 Subject: [Patches] [ python-Patches-907403 ] Improvements to cStringIO.writelines() Message-ID: Patches item #907403, was opened at 2004-03-01 04:35 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=907403&group_id=5470 Category: Modules Group: Python 2.4 >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Raymond Hettinger (rhettinger) Summary: Improvements to cStringIO.writelines() Initial Comment: Currently, cStringIO.writelines() only accepts a sequence argument and it passes that argument internally to str.join() to construct a large temporary string before writing. The patch allows the method to accept any iterable argument and for the lines to be written one at a time which cuts memory utilization in half. The result is especially valuable when used with generators or itertools because the strings are constructed on the fly rather than all at once. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 13:26 Message: Logged In: YES user_id=80475 Thanks for the second review. As requested: * Changed 'l' to 'n'. * Made a parallel change to StringIO.py. * Borrowed a docstring mentioning iterables from file.writelines. Checked-in as: Lib/StringIO.py 1.34 Modules/cStringIO.c 2.47 ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-07 22:25 Message: Logged In: YES user_id=33168 I don't see any problems with the new code. One nit though, I hate using l as a variable name because it's difficult to tell if it's an l (ell) or a 1 (one). I realize O_cwrite uses l. I'm not against n though. :-) Should the docstring be updated? It says the parameter is a sequence_of_strings. I'm not sure if it would be good to call it an iterable_of_strings. That sounds funny. I think it's good that it takes an iterable. That makes it more like StringIO. Should StringIO.writelines() be changed? It currently does: self.write(''.join(list)) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=907403&group_id=5470 From noreply at sourceforge.net Mon Mar 8 13:33:44 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 13:33:51 2004 Subject: [Patches] [ python-Patches-805830 ] Highlight builtins Message-ID: Patches item #805830, was opened at 2003-09-13 20:29 Message generated for change (Settings changed) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=805830&group_id=5470 Category: IDLE >Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Nigel Rowe (fisheggs) Assigned to: Kurt B. Kaiser (kbk) Summary: Highlight builtins Initial Comment: There was a recent discussion on c.l.p regarding shadowing builtins. (Summary, DONT). The point was made that decent editors should highlight builtins and keywords. The attached patch enables hightlighting of all entries in __builtin__ excluding those starting with a '_' ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2004-03-08 13:33 Message: Logged In: YES user_id=149084 Thanks for the patch! This also implemented a fix for IDLEfork bug [ 693418 ] Normal text background color not refreshed which is a significant contribution! Add a highlight theme for builtin keywords. Python Patch 805830 Nigel Rowe M ClassBrowser.py 1.8 M ColorDelegator.py 1.13 M EditorWindow.py 1.56 M NEWS.txt 1.31 M PyShell.py 1.87 M TreeWidget.py 1.9 M config-highlight.def 1.8 M configDialog.py 1.57 M configHandler.py 1.34 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=805830&group_id=5470 From noreply at sourceforge.net Mon Mar 8 15:35:17 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 15:35:24 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 13:22 Message generated for change (Comment added) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2004-03-08 20:35 Message: Logged In: YES user_id=4771 No, Neal's comments made me realize I didn't think about nested loops. In these, you have several iterators lying around on the stack on top of the list. Definitely not clean. I guess your patch is the cleanest so far. More subtle optimizations should be left to a bytecode optimizer. (btw I wonder why we don't consider these more.) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 17:51 Message: Logged In: YES user_id=80475 Armin, do you have a working patch (which handles nested list comps) that implements: LIST_APPEND [list, ignored, value] -> [list, ignored]. If you can get it to work, it would simplify and speedup the code a bit. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 17:38 Message: Logged In: YES user_id=33168 My last comment about "problems" referred to removing the DELETE_FAST by Armin. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 17:36 Message: Logged In: YES user_id=33168 You may have problems with inner list comps: [x+y for x in list1 for y in list2] If you only have a single list comp, and LIST_APPEND didn't pop the list off the stack, I think you could do: 0 BUILD_LIST 0 3 LOAD_FAST 0 (iter) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_FAST 2 (elem) 13 LIST_APPEND 14 JUMP_ABSOLUTE 7 >> 17 RETURN_VALUE ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 17:31 Message: Logged In: YES user_id=80475 The actual checkin kept the final DELETE_FAST to allow all references to the list to be removable. Will take a look at the proposed variants. Off-hand, they do not have a clean feel to them, but they do save a trip around the eval loop. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-08 10:58 Message: Logged In: YES user_id=4771 The patch also removes the final DELETE_FAST. Why? I think deleting this old reference to the list is a good idea. Another faster (but slightly less clear) approach would be to change the behavior of LIST_APPEND so that the '_' variable can be completely avoided. In stack manipulation notation: LIST_APPEND [list, ignored, value] -> [list, ignored] where 'ignored' is the iterator still on the stack. Admittedly this is quite an unexpected behavior, so maybe LIST_APPEND should be called LIST_COMPREHEND or somesuch. An intermediate solution with no change in LIST_APPEND would introduce a DUP_OVER stack manipulation opcode: DUP_OVER [a, b] -> [a, b, a] which could replace the LOAD_FAST '_' at the beginning of the loop, getting the list object from over the iterator. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 07:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Mon Mar 8 15:51:52 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 15:52:00 2004 Subject: [Patches] [ python-Patches-805830 ] Highlight builtins Message-ID: Patches item #805830, was opened at 2003-09-13 20:29 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=805830&group_id=5470 Category: IDLE Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Nigel Rowe (fisheggs) Assigned to: Kurt B. Kaiser (kbk) Summary: Highlight builtins Initial Comment: There was a recent discussion on c.l.p regarding shadowing builtins. (Summary, DONT). The point was made that decent editors should highlight builtins and keywords. The attached patch enables hightlighting of all entries in __builtin__ excluding those starting with a '_' ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2004-03-08 15:51 Message: Logged In: YES user_id=149084 This also fixed Python [897872 ] Unknown color name on HP-UX ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2004-03-08 13:33 Message: Logged In: YES user_id=149084 Thanks for the patch! This also implemented a fix for IDLEfork bug [ 693418 ] Normal text background color not refreshed which is a significant contribution! Add a highlight theme for builtin keywords. Python Patch 805830 Nigel Rowe M ClassBrowser.py 1.8 M ColorDelegator.py 1.13 M EditorWindow.py 1.56 M NEWS.txt 1.31 M PyShell.py 1.87 M TreeWidget.py 1.9 M config-highlight.def 1.8 M configDialog.py 1.57 M configHandler.py 1.34 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=805830&group_id=5470 From noreply at sourceforge.net Mon Mar 8 14:50:16 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 16:02:27 2004 Subject: [Patches] [ python-Patches-562100 ] Installation database patch Message-ID: Patches item #562100, was opened at 2002-05-29 17:21 Message generated for change (Comment added) made by fdrake You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=562100&group_id=5470 Category: Distutils and setup.py Group: None Status: Open Resolution: None Priority: 5 Submitted By: A.M. Kuchling (akuchling) Assigned to: A.M. Kuchling (akuchling) Summary: Installation database patch Initial Comment: The attached patch implements an installation database. This patch is not yet correct; don't bother proofreading it yet. I simply wanted to create a patch number so I could put it in PEP 262. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2004-03-08 14:50 Message: Logged In: YES user_id=3066 Just a note: The install_db.py file is in Python's CVS, in nondist/sandbox/pep262/. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2004-02-24 18:02 Message: Logged In: YES user_id=3066 The attached patch implies the addition of the file install_db.py, but does not include that file. Please attach that file to this patch report as well. Thanks! ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2004-01-30 12:29 Message: Logged In: YES user_id=11375 Reviving this patch. ---------------------------------------------------------------------- Comment By: Bob Ippolito (etrepum) Date: 2003-10-03 16:19 Message: Logged In: YES user_id=139309 This should be brought back into play (patch finished, PEP heavily considered, or however that goes), the Package Manager effort going on in pythonmac-sig needs this functionality as part of distutils. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2002-10-24 15:13 Message: Logged In: YES user_id=11375 Closing this patch with the withdrawal of PEP 262. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=562100&group_id=5470 From noreply at sourceforge.net Mon Mar 8 17:48:07 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 17:48:13 2004 Subject: [Patches] [ python-Patches-910929 ] Optimize list comprehensions Message-ID: Patches item #910929, was opened at 2004-03-06 08:22 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Nobody/Anonymous (nobody) Summary: Optimize list comprehensions Initial Comment: Save about 35% on the per pass overhead of list comprehensions. Adds a new opcode, LIST_APPEND, which is faster than the current call to CALL_FUNCTION 1 on the bound method, list.append(), and the subsequent call to POP_TOP to clear the returned None value. The resulting disassembled code is suprisingly light and concise. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 17:48 Message: Logged In: YES user_id=80475 I am currently working on building out the byte code optimizer. Let me know if you would like to participate. The three infrastructure components needed for further buildouts are: * Check a range to see if it is a basic block * Allow a tempory NOP code to be eliminated on a final pass which makes jump target fixups. * Fixup all of the line number references. I made the first two but not the last. For example, A basic block containing: [BUILD_TUPLE 2 UNPACK_SEQUENCE 2] or [BUILD_LIST 2 UNPACK_SEQUENCE 2] should be directly replaced with [ROT_TWO NOP NOP NOP NOP NOP] and be compressed on a final pass to [ROT_TWO] This gives a three fold speedup for: a,b=b,a The other candidate transformations are: [BUILD_TUPLE 3 UNPACK_SEQUENCE 3] --> [ROT_THREE ROT_TWO] [LOAD_CONST 2 BINARY_MULTIPLY] --> [DUP BINARY_ADD] [RETURN_VALUE anyLoadInstruction RETURN_vALUE] --> [RETURN_VALUE] [STORE_FAST n LOAD_FAST n] --> [DUP STORE_FAST n] [UNARY_NOT JUMP_IF_FALSE tgt POP_TOP ... tgt: POP_TOP] --> [JUMP_IF_TRUE tgt POP_TOP ... tgt: POP_TOP] [UNARY_NOT JUMP_IF_TRUE tgt POP_TOP ... tgt: POP_TOP] --> [JUMP_IF_FALSE tgt POP_TOP ... tgt: POP_TOP] ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-08 15:35 Message: Logged In: YES user_id=4771 No, Neal's comments made me realize I didn't think about nested loops. In these, you have several iterators lying around on the stack on top of the list. Definitely not clean. I guess your patch is the cleanest so far. More subtle optimizations should be left to a bytecode optimizer. (btw I wonder why we don't consider these more.) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 12:51 Message: Logged In: YES user_id=80475 Armin, do you have a working patch (which handles nested list comps) that implements: LIST_APPEND [list, ignored, value] -> [list, ignored]. If you can get it to work, it would simplify and speedup the code a bit. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 12:38 Message: Logged In: YES user_id=33168 My last comment about "problems" referred to removing the DELETE_FAST by Armin. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-08 12:36 Message: Logged In: YES user_id=33168 You may have problems with inner list comps: [x+y for x in list1 for y in list2] If you only have a single list comp, and LIST_APPEND didn't pop the list off the stack, I think you could do: 0 BUILD_LIST 0 3 LOAD_FAST 0 (iter) 6 GET_ITER >> 7 FOR_ITER 7 (to 17) 10 STORE_FAST 2 (elem) 13 LIST_APPEND 14 JUMP_ABSOLUTE 7 >> 17 RETURN_VALUE ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-08 12:31 Message: Logged In: YES user_id=80475 The actual checkin kept the final DELETE_FAST to allow all references to the list to be removable. Will take a look at the proposed variants. Off-hand, they do not have a clean feel to them, but they do save a trip around the eval loop. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-08 05:58 Message: Logged In: YES user_id=4771 The patch also removes the final DELETE_FAST. Why? I think deleting this old reference to the list is a good idea. Another faster (but slightly less clear) approach would be to change the behavior of LIST_APPEND so that the '_' variable can be completely avoided. In stack manipulation notation: LIST_APPEND [list, ignored, value] -> [list, ignored] where 'ignored' is the iterator still on the stack. Admittedly this is quite an unexpected behavior, so maybe LIST_APPEND should be called LIST_COMPREHEND or somesuch. An intermediate solution with no change in LIST_APPEND would introduce a DUP_OVER stack manipulation opcode: DUP_OVER [a, b] -> [a, b, a] which could replace the LOAD_FAST '_' at the beginning of the loop, getting the list object from over the iterator. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-07 02:41 Message: Logged In: YES user_id=80475 Applied to: Python/ceval.c 2.379 Python/compile.c 2.299 Include/opcode.h 2.44 Lib/opcode.py 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=910929&group_id=5470 From noreply at sourceforge.net Mon Mar 8 20:20:05 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 20:20:08 2004 Subject: [Patches] [ python-Patches-912410 ] HTMLParser should support entities in attributes Message-ID: Patches item #912410, was opened at 2004-03-08 19:20 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912410&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Aaron Swartz (aaronsw) Assigned to: Nobody/Anonymous (nobody) Summary: HTMLParser should support entities in attributes Initial Comment: HTMLParser doesn't currently support entities in attributes, like this: foo This patch fixes that. Simply replace the unescape in HTMLParser.py with: import htmlentitydefs def unescape(self, s): def replaceEntities(s): s = s.groups()[0] if s[0] == "#": s = s[1:] if s[0] in ['x','X']: c = int(s[1:], 16) else: c = int(s) return unichr(c) else: return unichr(htmlentitydefs.name2codepoint[c]) return re.sub(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));", replaceEntities, s) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912410&group_id=5470 From noreply at sourceforge.net Mon Mar 8 20:21:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 20:21:11 2004 Subject: [Patches] [ python-Patches-912410 ] HTMLParser should support entities in attributes Message-ID: Patches item #912410, was opened at 2004-03-08 19:20 Message generated for change (Comment added) made by aaronsw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912410&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Aaron Swartz (aaronsw) Assigned to: Nobody/Anonymous (nobody) Summary: HTMLParser should support entities in attributes Initial Comment: HTMLParser doesn't currently support entities in attributes, like this: foo This patch fixes that. Simply replace the unescape in HTMLParser.py with: import htmlentitydefs def unescape(self, s): def replaceEntities(s): s = s.groups()[0] if s[0] == "#": s = s[1:] if s[0] in ['x','X']: c = int(s[1:], 16) else: c = int(s) return unichr(c) else: return unichr(htmlentitydefs.name2codepoint[c]) return re.sub(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));", replaceEntities, s) ---------------------------------------------------------------------- >Comment By: Aaron Swartz (aaronsw) Date: 2004-03-08 19:21 Message: Logged In: YES user_id=122141 Oops. The replacement function is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912410&group_id=5470 From noreply at sourceforge.net Mon Mar 8 20:21:47 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 20:21:50 2004 Subject: [Patches] [ python-Patches-912410 ] HTMLParser should support entities in attributes Message-ID: Patches item #912410, was opened at 2004-03-08 19:20 Message generated for change (Comment added) made by aaronsw You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912410&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Aaron Swartz (aaronsw) Assigned to: Nobody/Anonymous (nobody) Summary: HTMLParser should support entities in attributes Initial Comment: HTMLParser doesn't currently support entities in attributes, like this: foo This patch fixes that. Simply replace the unescape in HTMLParser.py with: import htmlentitydefs def unescape(self, s): def replaceEntities(s): s = s.groups()[0] if s[0] == "#": s = s[1:] if s[0] in ['x','X']: c = int(s[1:], 16) else: c = int(s) return unichr(c) else: return unichr(htmlentitydefs.name2codepoint[c]) return re.sub(r"&(#?[xX]?(?:[0-9a-fA-F]+|\w{1,8}));", replaceEntities, s) ---------------------------------------------------------------------- >Comment By: Aaron Swartz (aaronsw) Date: 2004-03-08 19:21 Message: Logged In: YES user_id=122141 Argh. Hopefully now. ---------------------------------------------------------------------- Comment By: Aaron Swartz (aaronsw) Date: 2004-03-08 19:21 Message: Logged In: YES user_id=122141 Oops. The replacement function is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912410&group_id=5470 From noreply at sourceforge.net Mon Mar 8 22:09:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 22:09:27 2004 Subject: [Patches] [ python-Patches-912452 ] add phi as a constant in math module Message-ID: Patches item #912452, was opened at 2004-03-08 19:09 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912452&group_id=5470 Category: Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Lars R. Damerow (ldamerow) Assigned to: Nobody/Anonymous (nobody) Summary: add phi as a constant in math module Initial Comment: This patch add phi, the good old Golden Ratio, to the math module. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912452&group_id=5470 From noreply at sourceforge.net Mon Mar 8 22:28:21 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 8 22:28:24 2004 Subject: [Patches] [ python-Patches-912462 ] latex compile error in current CVS Message-ID: Patches item #912462, was opened at 2004-03-09 12:28 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912462&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: George Yoshida (quiver) Assigned to: Nobody/Anonymous (nobody) Summary: latex compile error in current CVS Initial Comment: Since the last change of Doc/lib/libstdtypes.tex, latex2html fails. It looks like \end{description} is not in the place where it should be. I attached a patch to fix this. I've confirmed that the compile succeeds without any errors on RHL 9.0. Here is the last part of a traceback I got: == ! LaTeX Error: Lonely \item--perhaps a missing list environment. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1341 \item[(9)] \function{update()} accepts either another mapping object ? ! Emergency stop. ... l.1341 \item[(9)] \function{update()} accepts either another mapping object Try typing to proceed. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912462&group_id=5470 From noreply at sourceforge.net Tue Mar 9 00:52:39 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 9 00:52:46 2004 Subject: [Patches] [ python-Patches-912462 ] latex compile error in current CVS Message-ID: Patches item #912462, was opened at 2004-03-09 12:28 Message generated for change (Settings changed) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912462&group_id=5470 Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: George Yoshida (quiver) >Assigned to: Hye-Shik Chang (perky) Summary: latex compile error in current CVS Initial Comment: Since the last change of Doc/lib/libstdtypes.tex, latex2html fails. It looks like \end{description} is not in the place where it should be. I attached a patch to fix this. I've confirmed that the compile succeeds without any errors on RHL 9.0. Here is the last part of a traceback I got: == ! LaTeX Error: Lonely \item--perhaps a missing list environment. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1341 \item[(9)] \function{update()} accepts either another mapping object ? ! Emergency stop. ... l.1341 \item[(9)] \function{update()} accepts either another mapping object Try typing to proceed. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912462&group_id=5470 From noreply at sourceforge.net Tue Mar 9 01:01:57 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 9 01:02:02 2004 Subject: [Patches] [ python-Patches-912462 ] latex compile error in current CVS Message-ID: Patches item #912462, was opened at 2004-03-09 12:28 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912462&group_id=5470 Category: Documentation Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: George Yoshida (quiver) Assigned to: Hye-Shik Chang (perky) Summary: latex compile error in current CVS Initial Comment: Since the last change of Doc/lib/libstdtypes.tex, latex2html fails. It looks like \end{description} is not in the place where it should be. I attached a patch to fix this. I've confirmed that the compile succeeds without any errors on RHL 9.0. Here is the last part of a traceback I got: == ! LaTeX Error: Lonely \item--perhaps a missing list environment. See the LaTeX manual or LaTeX Companion for explanation. Type H for immediate help. ... l.1341 \item[(9)] \function{update()} accepts either another mapping object ? ! Emergency stop. ... l.1341 \item[(9)] \function{update()} accepts either another mapping object Try typing to proceed. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-09 15:01 Message: Logged In: YES user_id=55188 Committed in Doc/lib/libstdtypes.tex 1.152 Thank you! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912462&group_id=5470 From noreply at sourceforge.net Tue Mar 9 13:55:52 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 9 13:55:59 2004 Subject: [Patches] [ python-Patches-889813 ] making the version of SSL configurable when creating sockets Message-ID: Patches item #889813, was opened at 2004-02-03 11:28 Message generated for change (Comment added) made by adamg-work You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=889813&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: adam goucher (adamg-work) Assigned to: Nobody/Anonymous (nobody) Summary: making the version of SSL configurable when creating sockets Initial Comment: Currently, socket.ssl uses the SSLv23 method of negotiating an ssl socket. This method connects with SSLv2 HELO packets and will negotiate up to a higher level if possible. However, if SSLv2 is turned off completly at the other side of the socket, this negotiation will fail. I have extended socket.ssl() to include another optional parameter -- the SSLmethod which can be any of the openssl methods (SSLv2, SSLv23, SSLv3, TLSv1). Existing functionality is maintained by providing SSLv23 as the default. Affected files: Lib/socket.py - extension of the function Modules/_ssl.c - guts of the changes socketmodule.h - theres a reference on how to make ssl sockets, so I added the change there Has been tested on solaris my making an ssl connection to a server, as well has httplib.HTTPSConnection() ---------------------------------------------------------------------- >Comment By: adam goucher (adamg-work) Date: 2004-03-09 13:55 Message: Logged In: YES user_id=939860 I have recreated the patch againt the 2.4 snapshot of 20040308. I also included the doc change and made TLSv1 the default. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-02-07 03:06 Message: Logged In: YES user_id=21627 Ok. Your patch is inapplicable to 2.3, anyway, as it adds a new feature. Are you willing to revise your patch accordingly? Please also include patches to the documentation. ---------------------------------------------------------------------- Comment By: adam goucher (adamg-work) Date: 2004-02-06 11:00 Message: Logged In: YES user_id=939860 I think in 2.3.x I would leave it as SSLv23 in order to not break things going from .x to .y. I think it would be reasonable to make TLS the default in the 2.4 branch though as some breakage is moreaceptable to more people when going between bigger releases. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-02-04 17:37 Message: Logged In: YES user_id=21627 Would be reasonable to change the default to TLS? ---------------------------------------------------------------------- Comment By: adam goucher (adamg-work) Date: 2004-02-03 11:30 Message: Logged In: YES user_id=939860 one last try to have all the files uploaded to the patch ---------------------------------------------------------------------- Comment By: adam goucher (adamg-work) Date: 2004-02-03 11:29 Message: Logged In: YES user_id=939860 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=889813&group_id=5470 From noreply at sourceforge.net Wed Mar 10 22:18:42 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 10 22:18:51 2004 Subject: [Patches] [ python-Patches-805830 ] Highlight builtins Message-ID: Patches item #805830, was opened at 2003-09-13 20:29 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=805830&group_id=5470 Category: IDLE Group: Python 2.4 >Status: Open Resolution: Accepted Priority: 5 Submitted By: Nigel Rowe (fisheggs) Assigned to: Kurt B. Kaiser (kbk) Summary: Highlight builtins Initial Comment: There was a recent discussion on c.l.p regarding shadowing builtins. (Summary, DONT). The point was made that decent editors should highlight builtins and keywords. The attached patch enables hightlighting of all entries in __builtin__ excluding those starting with a '_' ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2004-03-10 22:18 Message: Logged In: YES user_id=149084 There's bug in the patch: if a builtin keyword is typed after a ' or " (without an intervening space), the comment attribute is removed from the quote and the rest of the line is colorized as if it were not quoted. ## builtin = r"([^\.]\b|^)" + any("BUILTIN", builtinlist) + r"\b" builtin = r"\b" + any("BUILTIN", builtinlist) + r"\b" seems to fix the symptom but the original obviously had a purpose (which I've been unable to decipher). ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2004-03-08 15:51 Message: Logged In: YES user_id=149084 This also fixed Python [897872 ] Unknown color name on HP-UX ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2004-03-08 13:33 Message: Logged In: YES user_id=149084 Thanks for the patch! This also implemented a fix for IDLEfork bug [ 693418 ] Normal text background color not refreshed which is a significant contribution! Add a highlight theme for builtin keywords. Python Patch 805830 Nigel Rowe M ClassBrowser.py 1.8 M ColorDelegator.py 1.13 M EditorWindow.py 1.56 M NEWS.txt 1.31 M PyShell.py 1.87 M TreeWidget.py 1.9 M config-highlight.def 1.8 M configDialog.py 1.57 M configHandler.py 1.34 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=805830&group_id=5470 From noreply at sourceforge.net Thu Mar 11 07:55:49 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 08:03:24 2004 Subject: [Patches] [ python-Patches-914096 ] Little (improvement and standarization) to asyncore. Message-ID: Patches item #914096, was opened at 2004-03-11 09:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914096&group_id=5470 Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: alejandro david weil (aweil) Assigned to: Nobody/Anonymous (nobody) Summary: Little (improvement and standarization) to asyncore. Initial Comment: changes: 1- uses add/del_channel/setup_socket functions. Adding calls to them instead of pasting its functionallity inside dispatcher's code. 2- added dispatcher's handle_connect_event() function. It's now, one place to overwrite when transparent wrapping dispatcher. And replaced code like: self.handle_connect() self.connected = 1 to: handle_connect_event() 3- looponce() function added. 2 allowes to implement ssldispatcher, with little work. And, of course, it seems to be usefull for anyone who wants to subclass dispatcher. 3 I used it for when I don't want to keep looping until connections are finished. Don't know if it's against the asyncore principies. Also, I decided to make this changes because I found things like: handle_connect() was called, and sometimes connected were setup to 1 before, and sometimes after the call. I'll expect these changes could make it work in a predecible way! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914096&group_id=5470 From noreply at sourceforge.net Thu Mar 11 08:56:18 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 08:56:23 2004 Subject: [Patches] [ python-Patches-812369 ] module shutdown procedure based on GC Message-ID: Patches item #812369, was opened at 2003-09-25 02:49 Message generated for change (Comment added) made by glchapman You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=812369&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Armin Rigo (arigo) Assigned to: Nobody/Anonymous (nobody) Summary: module shutdown procedure based on GC Initial Comment: This patches changes PyImport_Cleanup() in an attempt to make the module shutdown order more predictable in the presence of modules that import each other. Before it explicitely clears the globals of the modules, it relies on the GC to try to do it more cleanly. http://mail.python.org/pipermail/python-dev/2003-September/038238.html To prevent objects with __del__ methods from keeping whole modules alive I actually replace each module with a weak reference to it in sys.modules. This allows me to find modules that remain alive after a call to PyGC_Collect(), and then go back to the old technique of clearing their globals. Note that weak references to dead cycles containing objects with finalizers have a strange property in Python: if you use the weak reference again you can break the cycles, but the objects with finalizers still won't be collected because they are in gc.garbage. As this is exactly what occurs above, I clear the gc.garbage list explicitely before the final PyGC_Collect() call. I'm not sure exactly what this might do; could it release older objects that were never put in a module but that at some time were put in gc.garbage and whose cycles were later broken? If so, is it a good thing to release them now? (Would it make sense to clear gc.garbage and call gc.collect again from time to time to check if the objects are still in a cycle?) This patch does not change the behavior of module objects clearing their globals dictionary as soon as they are deallocated. This could be work investigating. This patch puts weak references (actually proxies) in sys.modules but only at shutdown time. Moure thoughts could be given towards allowing weak references during normal program execution as well. To do so we must ensure that 'import' statements return the real module, not the weak proxy object, which is more difficult than it first seems in the presence of packages. And finally -- this patch has not really been tested, apart from the fact that it passes the test suite. ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2004-03-11 04:56 Message: Logged In: YES user_id=86307 Not sure if this is a good idea, but I wonder if the extensions dictionary should be cleared (ie _PyImport_Fini called) sooner, possibly even before PyImport_Cleanup. This would allow garbage collection during PyImport_Cleanup to catch anything a C module might have created which is in a cycle with its module (through a bad design on my part, I recently discovered I had created just such a cycle). I suppose _PyImport_Fini is called when it is called because some code during PyImport_Cleanup might import a dynamic module, which would then create a new extensions dictionary if _PyImport_Fini had already been run. Perhaps a flag could be added so that _PyImport_FixupExtension would not try to add a module's dict to extensions if Python is currently shutting down. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-09-27 09:31 Message: Logged In: YES user_id=21627 I think clearing gc.garbage at shutdown time is a good idea; gc.garbage is mostly a debugging aid, and should be empty in production code. If it still contains objects at shutdown time, it is IMO reasonable to give them a chance to become collected, in case somebody broke their cycles. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=812369&group_id=5470 From hqa46nvmul at ya.com Thu Mar 11 03:44:24 2004 From: hqa46nvmul at ya.com (Seth Mcneal) Date: Thu Mar 11 10:46:37 2004 Subject: [Patches] bedpost Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040311/92145100/attachment.html From noreply at sourceforge.net Thu Mar 11 10:49:18 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 12:03:04 2004 Subject: [Patches] [ python-Patches-909005 ] asyncore fixes and improvements Message-ID: Patches item #909005, was opened at 2004-03-03 14:07 Message generated for change (Comment added) made by calvin You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Alexey Klimkin (klimkin) Assigned to: Nobody/Anonymous (nobody) Summary: asyncore fixes and improvements Initial Comment: Minor: * 0/1 for boolean values replaced with False/True. * (887279) Added handling of POLLPRI as POLLIN. POLLERR, POLLHUP, POLLNVAL are handled as exception event. handle_expt_event gets recent error from self.socket object and raises socket.error. * Default readable()/writable() returns False. * Added "map" parameter for file_dispatcher. * file_wrapper: removed "return" in close(), recv/read and send/write swapped because of their nature. * mac code for writable() removed. Manual for accept() on mac is similar to the one on linux. * Repeating exception changed from "raise socket.error, why" to raise. * Added connected/accepting/addr reset on close(). Initialization of variables moved to __init__. * close_all() now calls close for dispatcher object, EBADF treated as already closed socket/file. * Added channel id to "unhandled..." messages. Bugs: * Fixed bug (654766,889153): client never gets connected, nor errored. Connecting client gets writable event from select(), however, some client may want always be non writable. Such client may never get connected. The fix adds _readable() - always True for accepting and always False for connecting socket; and _writable() - always False for accepting and always True for connecting socket. This implies, that listening dispatcher's readable() and writable() will never be called. ("man accept" and "man connect" for non-blocking sockets). * Fixed bug: error handling after accept(). It's said, that accept can return EWOULDBLOCK even for readable socket. This mean, that even after handle_accept(), dispatcher's accept() still raise EWOULDBLOCK. New code does accept() itself and stores accepted socket in self.__pending_accept. If there was socket.error, it's treated as EWOULDBLOCK. dispatcher's accept returns self.__pending_accept and resets it to None. Features: * Added pending_read() and pending_write(). The functions helps to use dispatcher over non socket objects with buffering capabilities. In original dispatcher, if socket makes buffered read and some data is in buffer, entering asyncore.poll() doesn't finishes, since there is no data in real file/socket. This feature allow to use SSL socket, since the socket reads data by 16k chunks. ---------------------------------------------------------------------- Comment By: Wummel (calvin) Date: 2004-03-11 16:49 Message: Logged In: YES user_id=9205 There is no file attached! You have to click on the checkbox next to the upload filename. This is a Sourceforge annoyance :( ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 From noreply at sourceforge.net Thu Mar 11 12:34:16 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 12:34:21 2004 Subject: [Patches] [ python-Patches-914291 ] Fix readline for utf-8 locales Message-ID: Patches item #914291, was opened at 2004-03-11 18:34 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914291&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Michal Čihař (nijel) Assigned to: Nobody/Anonymous (nobody) Summary: Fix readline for utf-8 locales Initial Comment: When using utf-8 locales, readline doesn't work correctly if not set LC_CTYPE to corresponding value. This patch adds missing calls to setlocale (I'm not sure whether it is really needed in setup_readline, but it definitely is in call_readline. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914291&group_id=5470 From m391vlsb at yandex.ru Thu Mar 11 17:00:25 2004 From: m391vlsb at yandex.ru (Calvin Nixon) Date: Thu Mar 11 13:06:26 2004 Subject: [Patches] get the better job - buy a diploma today Message-ID: <1icxs50852b5$$2--6uxsx76@x16s5> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040311/0e38c261/attachment.html From noreply at sourceforge.net Thu Mar 11 13:45:18 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 13:45:22 2004 Subject: [Patches] [ python-Patches-914340 ] gzip.GzipFile to accept stream as fileobj. Message-ID: Patches item #914340, was opened at 2004-03-11 13:45 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Igor Belyi (belyi) Assigned to: Nobody/Anonymous (nobody) Summary: gzip.GzipFile to accept stream as fileobj. Initial Comment: When gzip.GzipFile is initialized with a fileobj which does not have tell() and seek() methods (non-rewinding stream) it throws exception. The interesting thing is that it doesn't have to. The following patch updates gzip.py to allow any stream with just a read() method to be used. This is helpful if you want to be able to do something like: gzip.GzipFile(fileobj=urllib.urlopen("file:///README.gz")).readlines() or use GzipFile with sys.stdin stream. But keep in mind that seek() and rewind() methond of the GzipFile() won't for such stream even with the patch. Igor ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 From noreply at sourceforge.net Thu Mar 11 13:46:42 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 13:46:50 2004 Subject: [Patches] [ python-Patches-889813 ] making the version of SSL configurable when creating sockets Message-ID: Patches item #889813, was opened at 2004-02-03 11:28 Message generated for change (Comment added) made by adamg-work You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=889813&group_id=5470 Category: Modules >Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: adam goucher (adamg-work) Assigned to: Nobody/Anonymous (nobody) Summary: making the version of SSL configurable when creating sockets Initial Comment: Currently, socket.ssl uses the SSLv23 method of negotiating an ssl socket. This method connects with SSLv2 HELO packets and will negotiate up to a higher level if possible. However, if SSLv2 is turned off completly at the other side of the socket, this negotiation will fail. I have extended socket.ssl() to include another optional parameter -- the SSLmethod which can be any of the openssl methods (SSLv2, SSLv23, SSLv3, TLSv1). Existing functionality is maintained by providing SSLv23 as the default. Affected files: Lib/socket.py - extension of the function Modules/_ssl.c - guts of the changes socketmodule.h - theres a reference on how to make ssl sockets, so I added the change there Has been tested on solaris my making an ssl connection to a server, as well has httplib.HTTPSConnection() ---------------------------------------------------------------------- >Comment By: adam goucher (adamg-work) Date: 2004-03-11 13:46 Message: Logged In: YES user_id=939860 modifying the version this applies to from 2.3 to 2.4 to reflect the new patch ---------------------------------------------------------------------- Comment By: adam goucher (adamg-work) Date: 2004-03-09 13:55 Message: Logged In: YES user_id=939860 I have recreated the patch againt the 2.4 snapshot of 20040308. I also included the doc change and made TLSv1 the default. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-02-07 03:06 Message: Logged In: YES user_id=21627 Ok. Your patch is inapplicable to 2.3, anyway, as it adds a new feature. Are you willing to revise your patch accordingly? Please also include patches to the documentation. ---------------------------------------------------------------------- Comment By: adam goucher (adamg-work) Date: 2004-02-06 11:00 Message: Logged In: YES user_id=939860 I think in 2.3.x I would leave it as SSLv23 in order to not break things going from .x to .y. I think it would be reasonable to make TLS the default in the 2.4 branch though as some breakage is moreaceptable to more people when going between bigger releases. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-02-04 17:37 Message: Logged In: YES user_id=21627 Would be reasonable to change the default to TLS? ---------------------------------------------------------------------- Comment By: adam goucher (adamg-work) Date: 2004-02-03 11:30 Message: Logged In: YES user_id=939860 one last try to have all the files uploaded to the patch ---------------------------------------------------------------------- Comment By: adam goucher (adamg-work) Date: 2004-02-03 11:29 Message: Logged In: YES user_id=939860 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=889813&group_id=5470 From noreply at sourceforge.net Thu Mar 11 14:09:45 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 14:09:49 2004 Subject: [Patches] [ python-Patches-914358 ] gzip.GzipFile to accept stream as fileobj. Message-ID: Patches item #914358, was opened at 2004-03-11 14:09 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914358&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Igor Belyi (belyi) Assigned to: Nobody/Anonymous (nobody) Summary: gzip.GzipFile to accept stream as fileobj. Initial Comment: When gzip.GzipFile is initialized with a fileobj which does not have tell() and seek() methods (non-rewinding stream) it throws exception. The interesting thing is that it doesn't have to. The following patch updates gzip.py to allow any stream with just a read() method to be used. This is helpful if you want to be able to do something like: gzip.GzipFile(fileobj=urllib.urlopen("file:///README.gz")).readlines() or use GzipFile with sys.stdin stream. But keep in mind that seek() and rewind() methond of the GzipFile() won't for such stream even with the patch. Igor ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914358&group_id=5470 From noreply at sourceforge.net Thu Mar 11 15:04:58 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 15:05:04 2004 Subject: [Patches] [ python-Patches-914340 ] gzip.GzipFile to accept stream as fileobj. Message-ID: Patches item #914340, was opened at 2004-03-11 13:45 Message generated for change (Comment added) made by belyi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 Category: Modules >Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Igor Belyi (belyi) Assigned to: Nobody/Anonymous (nobody) Summary: gzip.GzipFile to accept stream as fileobj. Initial Comment: When gzip.GzipFile is initialized with a fileobj which does not have tell() and seek() methods (non-rewinding stream) it throws exception. The interesting thing is that it doesn't have to. The following patch updates gzip.py to allow any stream with just a read() method to be used. This is helpful if you want to be able to do something like: gzip.GzipFile(fileobj=urllib.urlopen("file:///README.gz")).readlines() or use GzipFile with sys.stdin stream. But keep in mind that seek() and rewind() methond of the GzipFile() won't for such stream even with the patch. Igor ---------------------------------------------------------------------- >Comment By: Igor Belyi (belyi) Date: 2004-03-11 15:04 Message: Logged In: YES user_id=995711 Previous revision of the patch does not work correctly with mutliple compressed members in one stream. I've updated the patch file. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 From noreply at sourceforge.net Thu Mar 11 15:06:50 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 15:06:56 2004 Subject: [Patches] [ python-Patches-914358 ] gzip.GzipFile to accept stream as fileobj. Message-ID: Patches item #914358, was opened at 2004-03-11 14:09 Message generated for change (Comment added) made by belyi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914358&group_id=5470 Category: Modules Group: Python 2.3 >Status: Deleted Resolution: None Priority: 5 Submitted By: Igor Belyi (belyi) Assigned to: Nobody/Anonymous (nobody) Summary: gzip.GzipFile to accept stream as fileobj. Initial Comment: When gzip.GzipFile is initialized with a fileobj which does not have tell() and seek() methods (non-rewinding stream) it throws exception. The interesting thing is that it doesn't have to. The following patch updates gzip.py to allow any stream with just a read() method to be used. This is helpful if you want to be able to do something like: gzip.GzipFile(fileobj=urllib.urlopen("file:///README.gz")).readlines() or use GzipFile with sys.stdin stream. But keep in mind that seek() and rewind() methond of the GzipFile() won't for such stream even with the patch. Igor ---------------------------------------------------------------------- >Comment By: Igor Belyi (belyi) Date: 2004-03-11 15:06 Message: Logged In: YES user_id=995711 This is a duplicated request caused by unintentional push of the "reload" button. I'm deleting it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914358&group_id=5470 From noreply at sourceforge.net Thu Mar 11 18:00:59 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 18:01:04 2004 Subject: [Patches] [ python-Patches-914546 ] Make history recall a-cyclic Message-ID: Patches item #914546, was opened at 2004-03-12 01:00 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914546&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Noam Raphael (noamr) Assigned to: Nobody/Anonymous (nobody) Summary: Make history recall a-cyclic Initial Comment: This patch adds an option for idleHistory to be a-cyclic -- holding the Alt-P keys will bring all the commands typed, one by one, and stop at the first one. This, I think, is more intuitive - when history is cyclic, and you press Alt-P continuously, you have no sense of your position, since it's always jumping from end to beginning. That's why I added this option, and that's why I made it a-cyclic by default. The diffs are against idlefork 0.9b1, but work against current idle cvs. Noam ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914546&group_id=5470 From noreply at sourceforge.net Thu Mar 11 18:50:27 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 11 18:50:40 2004 Subject: [Patches] [ python-Patches-914575 ] difflib side by side diff support, diff.py s/b/s HTML option Message-ID: Patches item #914575, was opened at 2004-03-11 17:50 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470 Category: Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Dan Gass (dmgass) Assigned to: Nobody/Anonymous (nobody) Summary: difflib side by side diff support, diff.py s/b/s HTML option Initial Comment: lib/difflib.py: Added support for generating side by side differences. Intended to be used for generating HTML pages but is generic where it can be used for other types of markup. tools/scripts/diff.py: Added -m option to use above patch to generate an HTML page of side by side differences between two files. The existing -c option when used with the new -m option controls whether contextual differences are shown or whether the entire file is displayed (number of context lines is controlled by existing -l option). NOTES: (1) Textual context diffs were included as requested. In addition, full and contextual HTML side by side differences (that this patch generated) are also included to assist in analyzing the differences and showing an example. (2) If this functionality is worthy of inclusion in the standard python distribution, I am willing to provide more documentation or make modifications to change the interface or make improvements. (3) When using Internet Explorer some font sizes seem to skew things a bit. Generally I've found the "smallest" to work best. If someone knows why, I'd be interested in making any necessary adjustments in the generated HTML. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:26:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 15:28:37 2004 Subject: [Patches] [ python-Patches-736962 ] Port tests to unittest (Part 2) Message-ID: Patches item #736962, was opened at 2003-05-13 05:45 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=736962&group_id=5470 Category: Tests Group: None Status: Open Resolution: Accepted Priority: 5 Submitted By: Walter D?rwald (doerwalter) Assigned to: Raymond Hettinger (rhettinger) Summary: Port tests to unittest (Part 2) Initial Comment: Here are the next test scripts ported to PyUnit: test_winsound and test_array. For test_array many additional tests have been added (code coverage is at 91%) ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 15:26 Message: Logged In: YES user_id=80475 Okay, this one is fine. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-18 18:49 Message: Logged In: YES user_id=89016 Here's the next one: test_binascii.py. Code coverage in binascii.c is at 92% (could be improved by enhancing test_binhex.py). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-13 17:46 Message: Logged In: YES user_id=33168 Looked good to me, test_future_pyunit.diff checked in as: * Lib/test/badsyntax_future3.py 1.2 * Lib/test/badsyntax_future4.py 1.2 * Lib/test/badsyntax_future5.py 1.2 * Lib/test/badsyntax_future6.py 1.2 * Lib/test/badsyntax_future7.py 1.2 * Lib/test/badsyntax_future8.py 1.1 * Lib/test/badsyntax_future9.py 1.1 * Lib/test/test_future.py 1.7 * Lib/test/test_future1.py 1.3 * Lib/test/test_future2.py 1.2 * Lib/test/output/test_future 1.4 ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-13 16:47 Message: Logged In: YES user_id=33168 Walter, I didn't get a mail from SF either. After you do a cvs add, you need to use -N to cvs diff. For example, cvs diff -N new_file_added_to_cvs_but_not_committed. If you look carefully, you can see this in the patch on the diff line for each file. I'll take a look at the patch. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-13 15:18 Message: Logged In: YES user_id=89016 Here is a version of test_future ported to PyUnit (test_future_pyunit.diff). If this makes sense to you Neal, please check it in. (BTW, how did you convince CVS to include badsyntax_future8.py and badsyntax_future9.py in the diff? Doing a "cvs add" only results in " Lib/test/badsyntax_future8.py is a new entry, no comparison available") ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-11 07:36 Message: Logged In: YES user_id=89016 md5/weakref patch checked in as: Lib/test/test_weakref.py 1.33 Lib/test/test_md5.py 1.5 Lib/test/output/test_md5 remove ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-11 01:05 Message: Logged In: YES user_id=33168 Improve coverage for test_future too. I'm not sure if test future can be ported to PyUnit. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-11 00:52 Message: Logged In: YES user_id=33168 Attached are two tests (in one file, I'm being lazy, sorry) to improve test coverage for md5c.c and _weakref.c. test_md5 was ported to unittest, weakref, just adds two little tests to get coverage up to 100%. If you like, just go ahead and check in. You will need to remove Lib/test/output/test_md5 ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-08 06:42 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/list_tests.py 1.1 Lib/test/seq_tests.py 1.1 Lib/test/test_list.py 1.1 Lib/test/test_tuple.py 1.1 Lib/test/test_types.py 1.56 Lib/test/test_userlist.py 1.11 Lib/test/output/test_types 1.3 Next will be test_binascii.py before I continue with test_types.py. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-12-07 18:49 Message: Logged In: YES user_id=80475 Looks good. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-11-27 14:48 Message: Logged In: YES user_id=89016 Here the first part of the test_types port: list and tuple tests have been moved to their own scripts: test_tuple.py and test_list.py. Common tests for tuple, list and UserList are shared (in seq_test.py and list_test.py) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-09-01 20:53 Message: Logged In: YES user_id=80475 Applied as: Lib/test/test_slice.py 1.5. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-31 18:52 Message: Logged In: YES user_id=80475 Looks good. I added a couple of minor tests. Revised patch attached. Okay to apply. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-31 14:49 Message: Logged In: YES user_id=89016 Thanks! Here's another one: test_slice.py ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-30 18:04 Message: Logged In: YES user_id=80475 Done. See: Lib/test/test_longexp.py 1.9; previous revision: 1.8 Lib/test/test_pep263.py 1.3; previous revision: 1.2 Lib/test/test_sets.py 1.27; previous revision: 1.26 Lib/test/test_structseq.py 1.5; previous revision: 1.4 Lib/test/output/test_longexp delete; previous revision: 1.3 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-30 12:10 Message: Logged In: YES user_id=80475 Will do! ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-30 12:06 Message: Logged In: YES user_id=89016 Here's test_structse.py converted with a few additional tests. If all three scripts are OK, could you check them in Raymond, as I'll be on vacation for three weeks? ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-09 10:47 Message: Logged In: YES user_id=89016 Here are two simple ones: test_pep263 and test_longexp. I can't think of any additional tests to add. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-23 08:38 Message: Logged In: YES user_id=80475 Fixed Walter's review comments and committed as Lib/test/test_compile.py 1.19 ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-23 06:49 Message: Logged In: YES user_id=89016 Are you sure, that the code path is the same in the new test_argument_handling() as in the old test? I.e. is "eval('lambda a,a: 0')" the same as "exec 'def f(a, a): pass'"? The print statement in test_float_literals should be changed to a comment. Should the imports in test_unary_minus() be moved to the start of the script? Otherwise the test looks (and runs) OK. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-20 14:26 Message: Logged In: YES user_id=80475 Here's one for you: test_compile.py is ready. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-19 04:46 Message: Logged In: YES user_id=89016 Maybe test_types.py should be split into several scripts: test_dict, test_tuple, test_list, test_int, test_long etc. Some of them already exist (like test_long), some don't. The constructor tests from test_builtin should probably be moved to the new test scripts as well. Furthermore we should try to share as much testing functionality as possible (e.g. between test_int and test_long, or between test_list and test_userlist) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-18 14:48 Message: Logged In: YES user_id=80475 Great! Can I suggest that test_types.py be next. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-18 09:27 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/test_builtin.py 1.21 Lib/test/test_complex.py 1.10 (I've moved the constructor tests from test_builtin to test_complex) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-17 18:49 Message: Logged In: YES user_id=80475 I added a few tests. If they are fine with you, go ahead and commit. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-17 16:36 Message: Logged In: YES user_id=89016 Here's the next one: text_complex.py. There one test that's currently commented out, because it crashes Python on Alpha (see http://www.python.org/sf/756093. The old test scripts states that tests for the constructor are in test_builtin, but I've added many tests to this script, so this is no longer true. We could move the tests to test_builtin, but IMHO that doesn't make sense, we'd better move the rest of the constructor tests from test_builtin to test_complex. I'd like to have a version of assertAlmostEqual() in unittest.py that can cope with complex numbers, but this would have to be coordinated with the standalone version of PyUnit (and it would probably have to wait until the 2.4 cycle starts) (I noticed that there is no assertAlmostEqual in the code on pyunit.sf.net anyway.) ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-17 07:06 Message: Logged In: YES user_id=89016 I'd like to keep this patch open, as it is an ongoing task (the next test scripts to be converted will be test_complex and then maybe test_marshal) ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-06-16 16:55 Message: Logged In: YES user_id=357491 OK, all tests pass cleanly. Applied as revision 1.6. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 14:51 Message: Logged In: YES user_id=89016 The joys of unittesting: Breaking code to make it better! ;) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 14:41 Message: Logged In: YES user_id=80475 Okay, it runs fine here. Once Brett confirms that it runs on the Mac, go ahead and load it. P.S. Your improved test_mimetools.py helped detect a latent error in mimetools.py when it was run under Windows. Tim made the fix this weekend. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 14:33 Message: Logged In: YES user_id=89016 Strange, I didn't get this failure here, but I got it on my laptop at home. I've removed the comparison with the getatime() value from test_time(). I hope this fixes it. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 14:09 Message: Logged In: YES user_id=80475 Walter, there is one failure left: ====================================== ================================ FAIL: test_time (__main__.PosixPathTest) ------------------------------------------------------------------- --- Traceback (most recent call last): File "test_posixpath.py", line 125, in test_time self.assert_( File "C:\PY23\lib\unittest.py", line 268, in failUnless if not expr: raise self.failureException, msg AssertionError Brett, after Walter revises the patch, just load the patch and make sure the test runs on the Mac. Between the three of us, we can validate the suite on three different platforms. Cheers. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-06-16 13:20 Message: Logged In: YES user_id=357491 Just wanting to work me like a dog, huh, Raymond? =) And to clarify for my and Walter's benefit, when you say guards, you mean that the tests don't crap out and say they failed on Windows, right? I thought posixpath was not meant to work under Windows. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 13:09 Message: Logged In: YES user_id=89016 I didn't realize that test_posixpath must work on Windows too. Here's a new version. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 11:04 Message: Logged In: YES user_id=80475 The previous comment applied to another patch. It should have said: Assigning to Brett to make sure the patch runs on the Mac. Don't accept this one until it has guards that allow the tests to run on Windows. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 10:59 Message: Logged In: YES user_id=80475 Assigning to Brett to give experience doing a detail review on this type of change. * examine every line of the diff and consider whether there is any semantic change (exceptions raised, etc). * apply the diff and run the test suite * in the interactive mode, call-up each function and make sure it behaves as expected (this is necessary because the test coverage is very low). * verify that the whitespace has been cleaned up. * look for missing changes (such as use of +=) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 10:44 Message: Logged In: YES user_id=80475 The test file now has dependencies that do not apply to windows. The failure messages are attached. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 07:48 Message: Logged In: YES user_id=89016 Here's the next one: test_posixpath.py with many additional tests. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-22 12:33 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/output/test_mimetools delete Lib/test/test_mimetools.py 1.4 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-22 11:18 Message: Logged In: YES user_id=80475 test_mimetools.py is ready. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-22 10:05 Message: Logged In: YES user_id=89016 I've attached a third version of test_mimetools.py that does some checks for the mimetools.Message class. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-21 08:04 Message: Logged In: YES user_id=80475 Attaching a slightly modified test_mimetools which covers more encodings and has a stronger set test. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-18 18:46 Message: Logged In: YES user_id=89016 Agreed, this is too much magic for too little gain. Back to business: Here is test_mimetools ported to PyUnit. Tests for mimetools.Message are still missing. If you can think of any tests please add them. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 22:18 Message: Logged In: YES user_id=80475 Get the module with sys.modules: tests = test_support.findtestclasses(sys.modules [__name__]) test_support.unittest(*tests) Yeah, the inheritance thing is a problem. I was trying to avoid having to modify unittest.TestCase to have a metaclass. The control of the module is kept in a separate SF project and one of its goals is to be backward compatible through 1.5.2 (meaning no metaclasses). A possible workaround is to define a modified testcase in test_support so that people don't import unittest directly anymore: test_support.py ------------------------- import unittest class SmartTestCase(unittest.TestCase): __metaclass__ = autotracktests pass test_sets.py ------------------ class TestBasicOps(test_support.SmartTestCase): run = False . . . class TestBasicOpsEmpty(TestBasicOps): def setUp(self): . . . Still, this is starting to seem a bit magical and tricky. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-17 21:52 Message: Logged In: YES user_id=89016 But how do I pass the module object from inside the module? And skipping abstract classes seems to be more work in this version: If skipping is done via a class attribute, derived classes have to explicitely reset this flag because of interitance. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 20:59 Message: Logged In: YES user_id=80475 Good call. Instead of using metaclasses, perhaps add a module introspector function to test_support: def findtestclasses(mod): tests = [] for elem in dir(mod): member = getattr(mod, elem) if type(member) != type: continue if issubclass(member, unittest.TestCase): tests.append(member) return tests ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-17 20:45 Message: Logged In: YES user_id=89016 But this can be solved with a special non-inheritable class attribute: class BaseTest(unittest.TestCase): run = False Then the metaclass can do the following: def __new__(cls, name, bases, dict): if "run" not in dict: dict["run"] = True cls = type.__new__(cls, name, bases, dict) if cls.run: tests.append(cls) return cls ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 20:04 Message: Logged In: YES user_id=80475 I don't think metaclasses or module introspection would help whenever there are classes that derive from TestCase but are not meant to be run directly (their subclasses have the setup/teardown/or class data). test_sets.py has examples of that kind of thing. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-17 19:50 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/test_array.py 1.20 Lib/test/test_winsound.py 1.5 Lib/test/output/test_winsound delete > The approach of using tests.append() is elegant and > makes it easier to verify that no tests are being omitted. The most elegant approach would probably be a metaclass that collects all TestCase subclasses that get defined. Classes that only serve as a base class could be skipped by specifying a certain class attribute. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 18:35 Message: Logged In: YES user_id=80475 The approach of using tests.append() is elegant and makes it easier to verify that no tests are being omitted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=736962&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:26:33 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 15:28:39 2004 Subject: [Patches] [ python-Patches-736962 ] Port tests to unittest (Part 2) Message-ID: Patches item #736962, was opened at 2003-05-13 05:45 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=736962&group_id=5470 Category: Tests Group: None Status: Open Resolution: Accepted Priority: 5 Submitted By: Walter D?rwald (doerwalter) >Assigned to: Walter D?rwald (doerwalter) Summary: Port tests to unittest (Part 2) Initial Comment: Here are the next test scripts ported to PyUnit: test_winsound and test_array. For test_array many additional tests have been added (code coverage is at 91%) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 15:26 Message: Logged In: YES user_id=80475 Okay, this one is fine. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-18 18:49 Message: Logged In: YES user_id=89016 Here's the next one: test_binascii.py. Code coverage in binascii.c is at 92% (could be improved by enhancing test_binhex.py). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-13 17:46 Message: Logged In: YES user_id=33168 Looked good to me, test_future_pyunit.diff checked in as: * Lib/test/badsyntax_future3.py 1.2 * Lib/test/badsyntax_future4.py 1.2 * Lib/test/badsyntax_future5.py 1.2 * Lib/test/badsyntax_future6.py 1.2 * Lib/test/badsyntax_future7.py 1.2 * Lib/test/badsyntax_future8.py 1.1 * Lib/test/badsyntax_future9.py 1.1 * Lib/test/test_future.py 1.7 * Lib/test/test_future1.py 1.3 * Lib/test/test_future2.py 1.2 * Lib/test/output/test_future 1.4 ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-13 16:47 Message: Logged In: YES user_id=33168 Walter, I didn't get a mail from SF either. After you do a cvs add, you need to use -N to cvs diff. For example, cvs diff -N new_file_added_to_cvs_but_not_committed. If you look carefully, you can see this in the patch on the diff line for each file. I'll take a look at the patch. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-13 15:18 Message: Logged In: YES user_id=89016 Here is a version of test_future ported to PyUnit (test_future_pyunit.diff). If this makes sense to you Neal, please check it in. (BTW, how did you convince CVS to include badsyntax_future8.py and badsyntax_future9.py in the diff? Doing a "cvs add" only results in " Lib/test/badsyntax_future8.py is a new entry, no comparison available") ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-11 07:36 Message: Logged In: YES user_id=89016 md5/weakref patch checked in as: Lib/test/test_weakref.py 1.33 Lib/test/test_md5.py 1.5 Lib/test/output/test_md5 remove ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-11 01:05 Message: Logged In: YES user_id=33168 Improve coverage for test_future too. I'm not sure if test future can be ported to PyUnit. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-11 00:52 Message: Logged In: YES user_id=33168 Attached are two tests (in one file, I'm being lazy, sorry) to improve test coverage for md5c.c and _weakref.c. test_md5 was ported to unittest, weakref, just adds two little tests to get coverage up to 100%. If you like, just go ahead and check in. You will need to remove Lib/test/output/test_md5 ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-08 06:42 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/list_tests.py 1.1 Lib/test/seq_tests.py 1.1 Lib/test/test_list.py 1.1 Lib/test/test_tuple.py 1.1 Lib/test/test_types.py 1.56 Lib/test/test_userlist.py 1.11 Lib/test/output/test_types 1.3 Next will be test_binascii.py before I continue with test_types.py. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-12-07 18:49 Message: Logged In: YES user_id=80475 Looks good. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-11-27 14:48 Message: Logged In: YES user_id=89016 Here the first part of the test_types port: list and tuple tests have been moved to their own scripts: test_tuple.py and test_list.py. Common tests for tuple, list and UserList are shared (in seq_test.py and list_test.py) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-09-01 20:53 Message: Logged In: YES user_id=80475 Applied as: Lib/test/test_slice.py 1.5. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-31 18:52 Message: Logged In: YES user_id=80475 Looks good. I added a couple of minor tests. Revised patch attached. Okay to apply. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-31 14:49 Message: Logged In: YES user_id=89016 Thanks! Here's another one: test_slice.py ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-30 18:04 Message: Logged In: YES user_id=80475 Done. See: Lib/test/test_longexp.py 1.9; previous revision: 1.8 Lib/test/test_pep263.py 1.3; previous revision: 1.2 Lib/test/test_sets.py 1.27; previous revision: 1.26 Lib/test/test_structseq.py 1.5; previous revision: 1.4 Lib/test/output/test_longexp delete; previous revision: 1.3 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-30 12:10 Message: Logged In: YES user_id=80475 Will do! ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-30 12:06 Message: Logged In: YES user_id=89016 Here's test_structse.py converted with a few additional tests. If all three scripts are OK, could you check them in Raymond, as I'll be on vacation for three weeks? ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-09 10:47 Message: Logged In: YES user_id=89016 Here are two simple ones: test_pep263 and test_longexp. I can't think of any additional tests to add. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-23 08:38 Message: Logged In: YES user_id=80475 Fixed Walter's review comments and committed as Lib/test/test_compile.py 1.19 ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-23 06:49 Message: Logged In: YES user_id=89016 Are you sure, that the code path is the same in the new test_argument_handling() as in the old test? I.e. is "eval('lambda a,a: 0')" the same as "exec 'def f(a, a): pass'"? The print statement in test_float_literals should be changed to a comment. Should the imports in test_unary_minus() be moved to the start of the script? Otherwise the test looks (and runs) OK. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-20 14:26 Message: Logged In: YES user_id=80475 Here's one for you: test_compile.py is ready. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-19 04:46 Message: Logged In: YES user_id=89016 Maybe test_types.py should be split into several scripts: test_dict, test_tuple, test_list, test_int, test_long etc. Some of them already exist (like test_long), some don't. The constructor tests from test_builtin should probably be moved to the new test scripts as well. Furthermore we should try to share as much testing functionality as possible (e.g. between test_int and test_long, or between test_list and test_userlist) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-18 14:48 Message: Logged In: YES user_id=80475 Great! Can I suggest that test_types.py be next. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-18 09:27 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/test_builtin.py 1.21 Lib/test/test_complex.py 1.10 (I've moved the constructor tests from test_builtin to test_complex) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-17 18:49 Message: Logged In: YES user_id=80475 I added a few tests. If they are fine with you, go ahead and commit. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-17 16:36 Message: Logged In: YES user_id=89016 Here's the next one: text_complex.py. There one test that's currently commented out, because it crashes Python on Alpha (see http://www.python.org/sf/756093. The old test scripts states that tests for the constructor are in test_builtin, but I've added many tests to this script, so this is no longer true. We could move the tests to test_builtin, but IMHO that doesn't make sense, we'd better move the rest of the constructor tests from test_builtin to test_complex. I'd like to have a version of assertAlmostEqual() in unittest.py that can cope with complex numbers, but this would have to be coordinated with the standalone version of PyUnit (and it would probably have to wait until the 2.4 cycle starts) (I noticed that there is no assertAlmostEqual in the code on pyunit.sf.net anyway.) ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-17 07:06 Message: Logged In: YES user_id=89016 I'd like to keep this patch open, as it is an ongoing task (the next test scripts to be converted will be test_complex and then maybe test_marshal) ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-06-16 16:55 Message: Logged In: YES user_id=357491 OK, all tests pass cleanly. Applied as revision 1.6. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 14:51 Message: Logged In: YES user_id=89016 The joys of unittesting: Breaking code to make it better! ;) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 14:41 Message: Logged In: YES user_id=80475 Okay, it runs fine here. Once Brett confirms that it runs on the Mac, go ahead and load it. P.S. Your improved test_mimetools.py helped detect a latent error in mimetools.py when it was run under Windows. Tim made the fix this weekend. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 14:33 Message: Logged In: YES user_id=89016 Strange, I didn't get this failure here, but I got it on my laptop at home. I've removed the comparison with the getatime() value from test_time(). I hope this fixes it. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 14:09 Message: Logged In: YES user_id=80475 Walter, there is one failure left: ====================================== ================================ FAIL: test_time (__main__.PosixPathTest) ------------------------------------------------------------------- --- Traceback (most recent call last): File "test_posixpath.py", line 125, in test_time self.assert_( File "C:\PY23\lib\unittest.py", line 268, in failUnless if not expr: raise self.failureException, msg AssertionError Brett, after Walter revises the patch, just load the patch and make sure the test runs on the Mac. Between the three of us, we can validate the suite on three different platforms. Cheers. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-06-16 13:20 Message: Logged In: YES user_id=357491 Just wanting to work me like a dog, huh, Raymond? =) And to clarify for my and Walter's benefit, when you say guards, you mean that the tests don't crap out and say they failed on Windows, right? I thought posixpath was not meant to work under Windows. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 13:09 Message: Logged In: YES user_id=89016 I didn't realize that test_posixpath must work on Windows too. Here's a new version. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 11:04 Message: Logged In: YES user_id=80475 The previous comment applied to another patch. It should have said: Assigning to Brett to make sure the patch runs on the Mac. Don't accept this one until it has guards that allow the tests to run on Windows. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 10:59 Message: Logged In: YES user_id=80475 Assigning to Brett to give experience doing a detail review on this type of change. * examine every line of the diff and consider whether there is any semantic change (exceptions raised, etc). * apply the diff and run the test suite * in the interactive mode, call-up each function and make sure it behaves as expected (this is necessary because the test coverage is very low). * verify that the whitespace has been cleaned up. * look for missing changes (such as use of +=) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 10:44 Message: Logged In: YES user_id=80475 The test file now has dependencies that do not apply to windows. The failure messages are attached. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 07:48 Message: Logged In: YES user_id=89016 Here's the next one: test_posixpath.py with many additional tests. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-22 12:33 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/output/test_mimetools delete Lib/test/test_mimetools.py 1.4 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-22 11:18 Message: Logged In: YES user_id=80475 test_mimetools.py is ready. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-22 10:05 Message: Logged In: YES user_id=89016 I've attached a third version of test_mimetools.py that does some checks for the mimetools.Message class. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-21 08:04 Message: Logged In: YES user_id=80475 Attaching a slightly modified test_mimetools which covers more encodings and has a stronger set test. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-18 18:46 Message: Logged In: YES user_id=89016 Agreed, this is too much magic for too little gain. Back to business: Here is test_mimetools ported to PyUnit. Tests for mimetools.Message are still missing. If you can think of any tests please add them. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 22:18 Message: Logged In: YES user_id=80475 Get the module with sys.modules: tests = test_support.findtestclasses(sys.modules [__name__]) test_support.unittest(*tests) Yeah, the inheritance thing is a problem. I was trying to avoid having to modify unittest.TestCase to have a metaclass. The control of the module is kept in a separate SF project and one of its goals is to be backward compatible through 1.5.2 (meaning no metaclasses). A possible workaround is to define a modified testcase in test_support so that people don't import unittest directly anymore: test_support.py ------------------------- import unittest class SmartTestCase(unittest.TestCase): __metaclass__ = autotracktests pass test_sets.py ------------------ class TestBasicOps(test_support.SmartTestCase): run = False . . . class TestBasicOpsEmpty(TestBasicOps): def setUp(self): . . . Still, this is starting to seem a bit magical and tricky. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-17 21:52 Message: Logged In: YES user_id=89016 But how do I pass the module object from inside the module? And skipping abstract classes seems to be more work in this version: If skipping is done via a class attribute, derived classes have to explicitely reset this flag because of interitance. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 20:59 Message: Logged In: YES user_id=80475 Good call. Instead of using metaclasses, perhaps add a module introspector function to test_support: def findtestclasses(mod): tests = [] for elem in dir(mod): member = getattr(mod, elem) if type(member) != type: continue if issubclass(member, unittest.TestCase): tests.append(member) return tests ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-17 20:45 Message: Logged In: YES user_id=89016 But this can be solved with a special non-inheritable class attribute: class BaseTest(unittest.TestCase): run = False Then the metaclass can do the following: def __new__(cls, name, bases, dict): if "run" not in dict: dict["run"] = True cls = type.__new__(cls, name, bases, dict) if cls.run: tests.append(cls) return cls ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 20:04 Message: Logged In: YES user_id=80475 I don't think metaclasses or module introspection would help whenever there are classes that derive from TestCase but are not meant to be run directly (their subclasses have the setup/teardown/or class data). test_sets.py has examples of that kind of thing. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-17 19:50 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/test_array.py 1.20 Lib/test/test_winsound.py 1.5 Lib/test/output/test_winsound delete > The approach of using tests.append() is elegant and > makes it easier to verify that no tests are being omitted. The most elegant approach would probably be a metaclass that collects all TestCase subclasses that get defined. Classes that only serve as a base class could be skipped by specifying a certain class attribute. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 18:35 Message: Logged In: YES user_id=80475 The approach of using tests.append() is elegant and makes it easier to verify that no tests are being omitted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=736962&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:27:36 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 15:29:17 2004 Subject: [Patches] [ python-Patches-914575 ] difflib side by side diff support, diff.py s/b/s HTML option Message-ID: Patches item #914575, was opened at 2004-03-11 18:50 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470 Category: Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Dan Gass (dmgass) >Assigned to: Tim Peters (tim_one) Summary: difflib side by side diff support, diff.py s/b/s HTML option Initial Comment: lib/difflib.py: Added support for generating side by side differences. Intended to be used for generating HTML pages but is generic where it can be used for other types of markup. tools/scripts/diff.py: Added -m option to use above patch to generate an HTML page of side by side differences between two files. The existing -c option when used with the new -m option controls whether contextual differences are shown or whether the entire file is displayed (number of context lines is controlled by existing -l option). NOTES: (1) Textual context diffs were included as requested. In addition, full and contextual HTML side by side differences (that this patch generated) are also included to assist in analyzing the differences and showing an example. (2) If this functionality is worthy of inclusion in the standard python distribution, I am willing to provide more documentation or make modifications to change the interface or make improvements. (3) When using Internet Explorer some font sizes seem to skew things a bit. Generally I've found the "smallest" to work best. If someone knows why, I'd be interested in making any necessary adjustments in the generated HTML. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914575&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:30:36 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 15:31:56 2004 Subject: [Patches] [ python-Patches-914096 ] Little (improvement and standarization) to asyncore. Message-ID: Patches item #914096, was opened at 2004-03-11 07:55 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914096&group_id=5470 Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: alejandro david weil (aweil) >Assigned to: A.M. Kuchling (akuchling) Summary: Little (improvement and standarization) to asyncore. Initial Comment: changes: 1- uses add/del_channel/setup_socket functions. Adding calls to them instead of pasting its functionallity inside dispatcher's code. 2- added dispatcher's handle_connect_event() function. It's now, one place to overwrite when transparent wrapping dispatcher. And replaced code like: self.handle_connect() self.connected = 1 to: handle_connect_event() 3- looponce() function added. 2 allowes to implement ssldispatcher, with little work. And, of course, it seems to be usefull for anyone who wants to subclass dispatcher. 3 I used it for when I don't want to keep looping until connections are finished. Don't know if it's against the asyncore principies. Also, I decided to make this changes because I found things like: handle_connect() was called, and sometimes connected were setup to 1 before, and sometimes after the call. I'll expect these changes could make it work in a predecible way! ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 15:30 Message: Logged In: YES user_id=80475 Andrew, is this module your baby now? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914096&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:40:37 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 15:41:29 2004 Subject: [Patches] [ python-Patches-911431 ] robot.txt must be robots.txt Message-ID: Patches item #911431, was opened at 2004-03-07 10:02 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911431&group_id=5470 Category: None >Group: Python 2.3 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: George Yoshida (quiver) Assigned to: Nobody/Anonymous (nobody) Summary: robot.txt must be robots.txt Initial Comment: There are some files in the source directory where 'robots.txt' is named 'robot.txt'(note the plural). As specified in norobots-rfc(*), the file name must be 'robots.txt'. * http://www.robotstxt.org/wc/norobots-rfc.html ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 15:40 Message: Logged In: YES user_id=80475 Accepted and applied. Thanks for the patch. See: Lib/robotparser.py 1.18 and 1.17.10.1 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=911431&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:41:27 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 15:42:52 2004 Subject: [Patches] [ python-Patches-909005 ] asyncore fixes and improvements Message-ID: Patches item #909005, was opened at 2004-03-03 08:07 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Alexey Klimkin (klimkin) >Assigned to: A.M. Kuchling (akuchling) Summary: asyncore fixes and improvements Initial Comment: Minor: * 0/1 for boolean values replaced with False/True. * (887279) Added handling of POLLPRI as POLLIN. POLLERR, POLLHUP, POLLNVAL are handled as exception event. handle_expt_event gets recent error from self.socket object and raises socket.error. * Default readable()/writable() returns False. * Added "map" parameter for file_dispatcher. * file_wrapper: removed "return" in close(), recv/read and send/write swapped because of their nature. * mac code for writable() removed. Manual for accept() on mac is similar to the one on linux. * Repeating exception changed from "raise socket.error, why" to raise. * Added connected/accepting/addr reset on close(). Initialization of variables moved to __init__. * close_all() now calls close for dispatcher object, EBADF treated as already closed socket/file. * Added channel id to "unhandled..." messages. Bugs: * Fixed bug (654766,889153): client never gets connected, nor errored. Connecting client gets writable event from select(), however, some client may want always be non writable. Such client may never get connected. The fix adds _readable() - always True for accepting and always False for connecting socket; and _writable() - always False for accepting and always True for connecting socket. This implies, that listening dispatcher's readable() and writable() will never be called. ("man accept" and "man connect" for non-blocking sockets). * Fixed bug: error handling after accept(). It's said, that accept can return EWOULDBLOCK even for readable socket. This mean, that even after handle_accept(), dispatcher's accept() still raise EWOULDBLOCK. New code does accept() itself and stores accepted socket in self.__pending_accept. If there was socket.error, it's treated as EWOULDBLOCK. dispatcher's accept returns self.__pending_accept and resets it to None. Features: * Added pending_read() and pending_write(). The functions helps to use dispatcher over non socket objects with buffering capabilities. In original dispatcher, if socket makes buffered read and some data is in buffer, entering asyncore.poll() doesn't finishes, since there is no data in real file/socket. This feature allow to use SSL socket, since the socket reads data by 16k chunks. ---------------------------------------------------------------------- Comment By: Wummel (calvin) Date: 2004-03-11 10:49 Message: Logged In: YES user_id=9205 There is no file attached! You have to click on the checkbox next to the upload filename. This is a Sourceforge annoyance :( ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:45:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 15:48:39 2004 Subject: [Patches] [ python-Patches-912452 ] add phi as a constant in math module Message-ID: Patches item #912452, was opened at 2004-03-08 22:09 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912452&group_id=5470 Category: Modules Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Lars R. Damerow (ldamerow) Assigned to: Nobody/Anonymous (nobody) Summary: add phi as a constant in math module Initial Comment: This patch add phi, the good old Golden Ratio, to the math module. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 15:45 Message: Logged In: YES user_id=80475 I appreciate the patch but need to reject it because this famous constant isn't essential for the math module or common floating point problems. In its few use cases, it can be computed directly. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=912452&group_id=5470 From noreply at sourceforge.net Sat Mar 13 15:55:10 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 16:07:50 2004 Subject: [Patches] [ python-Patches-906501 ] Fix typos in pystate.h comments Message-ID: Patches item #906501, was opened at 2004-02-28 10:44 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=906501&group_id=5470 Category: Core (C code) Group: Python 2.3 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Greg Chapman (glchapman) Assigned to: Raymond Hettinger (rhettinger) Summary: Fix typos in pystate.h comments Initial Comment: pystate.h has a couple of references to PyGILState_Acquire; these should refer to PyGILState_Ensure. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 15:55 Message: Logged In: YES user_id=80475 Accepted and applied as Include/pystate.h 2.26 Thanks for the patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=906501&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:08:29 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 16:09:31 2004 Subject: [Patches] [ python-Patches-873211 ] trace.py: simple bug in write_results_file Message-ID: Patches item #873211, was opened at 2004-01-08 13:07 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873211&group_id=5470 Category: Library (Lib) Group: None >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Bruno da Silva de Oliveira (nicoddemus) Assigned to: Nobody/Anonymous (nobody) Summary: trace.py: simple bug in write_results_file Initial Comment: The callers of write_results_file expect it to return a tuple (n_lines, n_hits), but right at the beginning of the function there's an attempt to open the results file, and if that fails it just "return"s, which causes an exception in the callers when they try to unpack the tuple. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:08 Message: Logged In: YES user_id=80475 The time machine beat you to this one. Guido already fixed this on Feb 19, 2004. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873211&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:16:28 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 16:17:16 2004 Subject: [Patches] [ python-Patches-873305 ] list.__setitem__(slice) behavior Message-ID: Patches item #873305, was opened at 2004-01-08 15:46 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Jonathan Brandmeyer (jbrandmeyer) Assigned to: Nobody/Anonymous (nobody) Summary: list.__setitem__(slice) behavior Initial Comment: Consider the following: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0] = [5, 6, 7] Currently the result is that x == [0,1,2,3,5,6,7,4]. However, I believe that calling setitem with an empty slice should be a no-op, rather than performing an insertion starting at the 'begin' index of the slice. The following patch to Objects/listobject.c makes this change. Thanks, Jonathan Brandmeyer ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:16 Message: Logged In: YES user_id=80475 I believe we are stuck with this one. Assigning to an empty slice like a[2:2]=list('hotdog') is a valid and useful. The weird part is that the rightmost index gets adjusted to make the most sense given the leftmost index. In your example, the offerred indices of -1 and 0 would seem to indicate a programming error. However, this autoadjusting is a long standing feature and occassionally useful in its own right. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:17:07 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 16:18:18 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 07:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Hye-Shik Chang (perky) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-23 11:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 08:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 05:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 01:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 06:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 04:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 07:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 07:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 06:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 03:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 00:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-29 22:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 06:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 02:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-26 10:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 01:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-25 18:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-21 13:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 09:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 06:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-11 22:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 20:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 00:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-07 23:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 07:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:19:37 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 16:20:30 2004 Subject: [Patches] [ python-Patches-869688 ] Add start and end optional args to array.index Message-ID: Patches item #869688, was opened at 2004-01-02 20:45 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=869688&group_id=5470 Category: Modules Group: Python 2.3 >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: L. Peter Deutsch (lpd) Assigned to: Nobody/Anonymous (nobody) Summary: Add start and end optional args to array.index Initial Comment: array.index should accept optional start and end args, like string.index and list.index. This patch adds this capability to the code, the test, and the documentation. I basically copied the code from list.index. This is my first Python patch. The submission guidelines suggested bundling the code, test, and doc patches together, so I did, even though there are separate Category entries for each of these. If I should have submitted them separately, please let me know. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:19 Message: Logged In: YES user_id=80475 The time machine beat you to this one. It was already implemented for Py2.4. See SF feature request #754014. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=869688&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:20:41 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 16:21:34 2004 Subject: [Patches] [ python-Patches-852334 ] Replace backticks with repr() Message-ID: Patches item #852334, was opened at 2003-12-01 16:35 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=852334&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: Fixed Priority: 5 Submitted By: Walter D?rwald (doerwalter) Assigned to: Anthony Baxter (anthonybaxter) Summary: Replace backticks with repr() Initial Comment: This patch removes most used of backticks in the standard library and replaces them with a call to repr() or uses '%r' in format string. I didn't touch the email package, the lib-old directory or test_grammar.py. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:20 Message: Logged In: YES user_id=80475 Can this be closed? ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2004-02-12 12:41 Message: Logged In: YES user_id=89016 Checked in. I've got a few smtplib.SMTPServerDisconnected exceptions when committing, but this was probably only for the commit email. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-02-06 13:52 Message: Logged In: YES user_id=80475 If you're feeling confident, go ahead and submit it. I don't think Anthony has time for additional review. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2004-02-06 13:29 Message: Logged In: YES user_id=89016 A new version of the patch that incorporates Gerrit's fixes can be found at http://styx.livinglogic.de/~walter/backtick2repr2.txt. The failure in test_doctest.py has disappeared. Is the patch ready to be checked in now? ---------------------------------------------------------------------- Comment By: Gerrit Holl (gerrit) Date: 2004-02-01 05:19 Message: Logged In: YES user_id=13298 Hmm, it doesn't seem to be possible to add an attachement to a tracker not sumbitted by yourself. I've now uploaded the metapatch to: http://people.nl.linux.org/~gerrit/creaties/b2r.metadiff ---------------------------------------------------------------------- Comment By: Gerrit Holl (gerrit) Date: 2004-02-01 05:10 Message: Logged In: YES user_id=13298 Raymond Hettinger asked me if I could do a second review for this patch, so I did. I found three errors. The first one is in Lib/lib-old/codehack.py. It's a mistake that the change is there at all. The second one in in Lib/plat-mac/aetypes.py where a 'r' is missing. The third and last one is in Tools/scripts/methfix.py. Here, a variable is printed twice. First one: - key = `co` # arbitrary but uniquely identifying string + key = co` # arbitrary but uniquely identifying string Second one: - return "QDRectangle(%s, %s, %s, %s)" % (`self.v0`, `self.h0`, - `self.v1`, `self.h1`) + return "QDRectangle(%r, %r, %r, %)" % (self.v0, self.h0, self.v1, self.h1) Third one: - err(tempname + ': warning: chmod failed (' + `msg` + ')\n') + err(tempname + '%s: warning: chmod failed (%r)\n' % (tempname, msg)) # Then make a backup of the original file as filename~ try: os.rename(filename, filename + '~') except os.error, msg: - err(filename + ': warning: backup failed (' + `msg` + ')\n') + err(filename + '%s: warning: backup failed (%r)\n' % (filename, msg)) # Now move the temp file to the original file try: os.rename(tempname, filename) except os.error, msg: - err(filename + ': rename failed (' + `msg` + ')\n') + err(filename + '%s: rename failed (%r)\n' % (filename, msg)) Attached is a meta-patch containing the differences between the original backticks2repr.txt which contains some errors, and the one with the errors removed. Note that I did not run any test suite or something like that, because I understand that had already been done. Other things I noticed: - sometimes this patch uses "foo %r bar" % baz, sometimes "foo %r bar" % (baz,), and sometimes "foo " + repr(baz) + " bar" - division needs a check as well, as '/' should be replaced by '//' in a lot of places. - there is one place where a (very) small behaviour change occurs, namely in Demo/sockets/gopher.py, where "print '(Bad line from server:', `line`, ')'" is replaced by "print '(Bad line from server: %r)' % (line,)", which is a difference of one whitespace character - I don't think it would cause a lot of trouble ;-) - it is possible that there are more errors in the patch which I didn't see, but it's also possible that there aren't any. (Hmm, I don't see immediatly how to add a patch...) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-01-01 02:31 Message: Logged In: YES user_id=80475 Any progress on getting this reviewed? ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-01 17:59 Message: Logged In: YES user_id=89016 Updated the patch so that the test suite works again (except for test_doctest.py) ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-01 16:38 Message: Logged In: YES user_id=89016 Oops, uploading the patch didn't work, as it's too big. It can be found at http://styx.livinglogic.de/~walter/backticks2repr.txt ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=852334&group_id=5470 From noreply at sourceforge.net Sat Mar 13 19:19:56 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 19:20:20 2004 Subject: [Patches] [ python-Patches-852334 ] Replace backticks with repr() Message-ID: Patches item #852334, was opened at 2003-12-01 22:35 Message generated for change (Settings changed) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=852334&group_id=5470 Category: Library (Lib) Group: Python 2.4 >Status: Closed Resolution: Fixed Priority: 5 Submitted By: Walter D?rwald (doerwalter) Assigned to: Anthony Baxter (anthonybaxter) Summary: Replace backticks with repr() Initial Comment: This patch removes most used of backticks in the standard library and replaces them with a call to repr() or uses '%r' in format string. I didn't touch the email package, the lib-old directory or test_grammar.py. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 22:20 Message: Logged In: YES user_id=80475 Can this be closed? ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2004-02-12 18:41 Message: Logged In: YES user_id=89016 Checked in. I've got a few smtplib.SMTPServerDisconnected exceptions when committing, but this was probably only for the commit email. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-02-06 19:52 Message: Logged In: YES user_id=80475 If you're feeling confident, go ahead and submit it. I don't think Anthony has time for additional review. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2004-02-06 19:29 Message: Logged In: YES user_id=89016 A new version of the patch that incorporates Gerrit's fixes can be found at http://styx.livinglogic.de/~walter/backtick2repr2.txt. The failure in test_doctest.py has disappeared. Is the patch ready to be checked in now? ---------------------------------------------------------------------- Comment By: Gerrit Holl (gerrit) Date: 2004-02-01 11:19 Message: Logged In: YES user_id=13298 Hmm, it doesn't seem to be possible to add an attachement to a tracker not sumbitted by yourself. I've now uploaded the metapatch to: http://people.nl.linux.org/~gerrit/creaties/b2r.metadiff ---------------------------------------------------------------------- Comment By: Gerrit Holl (gerrit) Date: 2004-02-01 11:10 Message: Logged In: YES user_id=13298 Raymond Hettinger asked me if I could do a second review for this patch, so I did. I found three errors. The first one is in Lib/lib-old/codehack.py. It's a mistake that the change is there at all. The second one in in Lib/plat-mac/aetypes.py where a 'r' is missing. The third and last one is in Tools/scripts/methfix.py. Here, a variable is printed twice. First one: - key = `co` # arbitrary but uniquely identifying string + key = co` # arbitrary but uniquely identifying string Second one: - return "QDRectangle(%s, %s, %s, %s)" % (`self.v0`, `self.h0`, - `self.v1`, `self.h1`) + return "QDRectangle(%r, %r, %r, %)" % (self.v0, self.h0, self.v1, self.h1) Third one: - err(tempname + ': warning: chmod failed (' + `msg` + ')\n') + err(tempname + '%s: warning: chmod failed (%r)\n' % (tempname, msg)) # Then make a backup of the original file as filename~ try: os.rename(filename, filename + '~') except os.error, msg: - err(filename + ': warning: backup failed (' + `msg` + ')\n') + err(filename + '%s: warning: backup failed (%r)\n' % (filename, msg)) # Now move the temp file to the original file try: os.rename(tempname, filename) except os.error, msg: - err(filename + ': rename failed (' + `msg` + ')\n') + err(filename + '%s: rename failed (%r)\n' % (filename, msg)) Attached is a meta-patch containing the differences between the original backticks2repr.txt which contains some errors, and the one with the errors removed. Note that I did not run any test suite or something like that, because I understand that had already been done. Other things I noticed: - sometimes this patch uses "foo %r bar" % baz, sometimes "foo %r bar" % (baz,), and sometimes "foo " + repr(baz) + " bar" - division needs a check as well, as '/' should be replaced by '//' in a lot of places. - there is one place where a (very) small behaviour change occurs, namely in Demo/sockets/gopher.py, where "print '(Bad line from server:', `line`, ')'" is replaced by "print '(Bad line from server: %r)' % (line,)", which is a difference of one whitespace character - I don't think it would cause a lot of trouble ;-) - it is possible that there are more errors in the patch which I didn't see, but it's also possible that there aren't any. (Hmm, I don't see immediatly how to add a patch...) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-01-01 08:31 Message: Logged In: YES user_id=80475 Any progress on getting this reviewed? ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-01 23:59 Message: Logged In: YES user_id=89016 Updated the patch so that the test suite works again (except for test_doctest.py) ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-01 22:38 Message: Logged In: YES user_id=89016 Oops, uploading the patch didn't work, as it's too big. It can be found at http://styx.livinglogic.de/~walter/backticks2repr.txt ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=852334&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:54:11 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 19:31:08 2004 Subject: [Patches] [ python-Patches-873305 ] list.__setitem__(slice) behavior Message-ID: Patches item #873305, was opened at 2004-01-08 15:46 Message generated for change (Comment added) made by jbrandmeyer You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 Category: None Group: None >Status: Open Resolution: Wont Fix Priority: 5 Submitted By: Jonathan Brandmeyer (jbrandmeyer) Assigned to: Nobody/Anonymous (nobody) Summary: list.__setitem__(slice) behavior Initial Comment: Consider the following: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0] = [5, 6, 7] Currently the result is that x == [0,1,2,3,5,6,7,4]. However, I believe that calling setitem with an empty slice should be a no-op, rather than performing an insertion starting at the 'begin' index of the slice. The following patch to Objects/listobject.c makes this change. Thanks, Jonathan Brandmeyer ---------------------------------------------------------------------- >Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:54 Message: Logged In: YES user_id=676765 Here is a snippet from a conversation that occured on the python-c++ list in January: > So now the question is, > what is the result of replacing an empty slice in a container with a > non-empty sequence? If the container supports insertion, inserting the > sequence seems logical enough to me. So how do you define the point to perform the insertion? If the user asks to replace every element that is greater than or equal to the fourth and less than the first element, where do you place the new sequence? The current action for a built-in list is to range-limit the stop point to be not less than the start point, but I think that is just for safety rather than an API decision, and that it is wrong. ---------------------------------------------------------------------- Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:42 Message: Logged In: YES user_id=676765 Consider this alternate example which should mean exactly the same thing, but doesn't: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0:1] = [5, 6, 7] Traceback (most recent call last): File "", line 1, in ? ValueError: attempt to assign sequence of size 3 to extended slice of size 0 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:16 Message: Logged In: YES user_id=80475 I believe we are stuck with this one. Assigning to an empty slice like a[2:2]=list('hotdog') is a valid and useful. The weird part is that the rightmost index gets adjusted to make the most sense given the leftmost index. In your example, the offerred indices of -1 and 0 would seem to indicate a programming error. However, this autoadjusting is a long standing feature and occassionally useful in its own right. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:42:55 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 20:06:40 2004 Subject: [Patches] [ python-Patches-873305 ] list.__setitem__(slice) behavior Message-ID: Patches item #873305, was opened at 2004-01-08 15:46 Message generated for change (Comment added) made by jbrandmeyer You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 Category: None Group: None Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Jonathan Brandmeyer (jbrandmeyer) Assigned to: Nobody/Anonymous (nobody) Summary: list.__setitem__(slice) behavior Initial Comment: Consider the following: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0] = [5, 6, 7] Currently the result is that x == [0,1,2,3,5,6,7,4]. However, I believe that calling setitem with an empty slice should be a no-op, rather than performing an insertion starting at the 'begin' index of the slice. The following patch to Objects/listobject.c makes this change. Thanks, Jonathan Brandmeyer ---------------------------------------------------------------------- >Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:42 Message: Logged In: YES user_id=676765 Consider this alternate example which should mean exactly the same thing, but doesn't: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0:1] = [5, 6, 7] Traceback (most recent call last): File "", line 1, in ? ValueError: attempt to assign sequence of size 3 to extended slice of size 0 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:16 Message: Logged In: YES user_id=80475 I believe we are stuck with this one. Assigning to an empty slice like a[2:2]=list('hotdog') is a valid and useful. The weird part is that the rightmost index gets adjusted to make the most sense given the leftmost index. In your example, the offerred indices of -1 and 0 would seem to indicate a programming error. However, this autoadjusting is a long standing feature and occassionally useful in its own right. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 From valcezar at ag0ny.com Sun Mar 14 14:12:27 2004 From: valcezar at ag0ny.com (Prince Cox) Date: Sat Mar 13 21:58:20 2004 Subject: [Patches] opportunity Message-ID: Diplomas from prestigious non-accredited universities NO required tests, classes, books, or interviews. Bachelors, Masters, MBA, and Doctorate (PhD) diplomas available in the field of your choice - that's right, you can become a Doctor and receive all the benefits and admiration that comes with it! No one is turned down. CALL TODAY ------->> 1 - 270-573-8493 (24 hours) <<----------- "I finally got my dream job that I knew I deserved. Thank you very much... " Fred Thompson, Kansas City, MO. "This got me a great job, that I never could have have without this program. Thank you. " Sharra Minnett , Chicago, IL From noreply at sourceforge.net Sat Mar 13 21:59:31 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 21:59:43 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Hye-Shik Chang (perky) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sat Mar 13 16:26:00 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 13 23:22:37 2004 Subject: [Patches] [ python-Patches-836879 ] assert should not generate code if optimized Message-ID: Patches item #836879, was opened at 2003-11-05 18:11 Message generated for change (Settings changed) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=836879&group_id=5470 Category: Library (Lib) Group: None Status: Open Resolution: None >Priority: 6 Submitted By: Neil Schemenauer (nascheme) Assigned to: Jeremy Hylton (jhylton) Summary: assert should not generate code if optimized Initial Comment: The compiler package in Lib still generates code for assert statements, even if -O is specified. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=836879&group_id=5470 From noreply at sourceforge.net Sun Mar 14 01:14:07 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 14 01:14:12 2004 Subject: [Patches] [ python-Patches-873305 ] list.__setitem__(slice) behavior Message-ID: Patches item #873305, was opened at 2004-01-08 15:46 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 Category: None Group: None Status: Open >Resolution: None Priority: 5 Submitted By: Jonathan Brandmeyer (jbrandmeyer) Assigned to: Nobody/Anonymous (nobody) Summary: list.__setitem__(slice) behavior Initial Comment: Consider the following: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0] = [5, 6, 7] Currently the result is that x == [0,1,2,3,5,6,7,4]. However, I believe that calling setitem with an empty slice should be a no-op, rather than performing an insertion starting at the 'begin' index of the slice. The following patch to Objects/listobject.c makes this change. Thanks, Jonathan Brandmeyer ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 01:14 Message: Logged In: YES user_id=80475 IMO, the risks of changing this outweight any possible benefits. I would be interested to see whether others would rally for or against this one. ---------------------------------------------------------------------- Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:54 Message: Logged In: YES user_id=676765 Here is a snippet from a conversation that occured on the python-c++ list in January: > So now the question is, > what is the result of replacing an empty slice in a container with a > non-empty sequence? If the container supports insertion, inserting the > sequence seems logical enough to me. So how do you define the point to perform the insertion? If the user asks to replace every element that is greater than or equal to the fourth and less than the first element, where do you place the new sequence? The current action for a built-in list is to range-limit the stop point to be not less than the start point, but I think that is just for safety rather than an API decision, and that it is wrong. ---------------------------------------------------------------------- Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:42 Message: Logged In: YES user_id=676765 Consider this alternate example which should mean exactly the same thing, but doesn't: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0:1] = [5, 6, 7] Traceback (most recent call last): File "", line 1, in ? ValueError: attempt to assign sequence of size 3 to extended slice of size 0 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:16 Message: Logged In: YES user_id=80475 I believe we are stuck with this one. Assigning to an empty slice like a[2:2]=list('hotdog') is a valid and useful. The weird part is that the rightmost index gets adjusted to make the most sense given the leftmost index. In your example, the offerred indices of -1 and 0 would seem to indicate a programming error. However, this autoadjusting is a long standing feature and occassionally useful in its own right. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 From g388ealj at cnt.ru Sun Mar 14 14:52:43 2004 From: g388ealj at cnt.ru (Ira Fields) Date: Sun Mar 14 14:55:36 2004 Subject: [Patches] Our portfolio winners have the highest percentage gains huea dceoait fta Message-ID: Investor Financial Times Report Specializing in Undervalued Small Cap Stocks for Immediate Breakout We have the #1 track record for our most recent recommendations in 2004: DLGI at .27 Currently .88 High 1.69 UP 526% SWYC at .18 Currently 1.38 High 1.98 UP 1000% FPDI at .21 Currently 1.08 High 1.25 UP 495% VDWB at .18 Currently 1.40 High 2.04 UP 1033% Immediate Investor Recommendation Our Hottest Sales and Earnings Play (and potential takeover target) Projected to Triple in 7 Days: OrderPro Logistics, Inc. (OTCBB: OPLO) Price--- .16 Sales '03--- over 2.3 million +2,700% growth over previous year Est. Sales '04--- over 10 million Average PE--- Industry 22-25 7 day target--- .58 30 day target--- .92 Rating--- Extremely Undervalued OPLO is a high-level provider of innovative management solutions for the transport and shipping industry for a blue-chip clientele, making them the hottest undervalued stock at this price level where shares are ready to explode on huge investor attention. Sales have rocketed beyond all estimates for OPLO over the last 12 months with no signs of slowing. The numbers continue to stack-up as present sales figures combined with current acquisition candidates, acquired and in process, total revenues of almost $40 million over the next 24 months. We are not the first to uncover this phenomenon as the stock is under accumulation, but we are acting aggressively on this recently filed data. Major clients include Sears, Office Max, Union Pacific Railroad, NordicTrack, Pacer Global (the logistics company for Ford and General Motors), along with many other large and mid-level corporate giants looking to benefit from the Company's expertise in transportation and supply chain management, freight brokerage services, packaging assessment, and private fleet management. OPLO can be considered a potential candidate to be acquired as their growth and suite of services matches up identically to many companies acquired by UPS and FedEx over the past few years. We are expecting many significant upcoming press releases regarding record-breaking revenues and the completion of extremely profitable acquisitions. OPLO is gaining in all the right categories with perhaps the one that matters most being the rapidly increasing attention from analysts, brokers, and aggressive investors with an eye for value and growth. OPLO has all the ingredients for major profits which is why we are seeing gains of 400% or more for early investors. This stock recommendation carries our highest rating for short-term trading profits. Investor Financial Times Report is an independent newsletter with the goal of giving investors the necessary knowledge to make rational and profitable investment decisions. This publication does not provide an analysis of the company's financial position and is not an offer to buy or sell securities Investing in securities is speculative and carries risk. It is recommended that any investment should be made after consulting with your investment advisor and after reviewing the financial statements of the company. Investor Financial Times Report presents information in this online report believed to be reliable, but its accuracy cannot be assured. Past performance does not insure similar future results. Investor Financial Times Report received four thousand dollars from an unaffiliated third party with respect to the preparation of this special online report as an effort to build investor awareness for OrderPro Logistics. The information reported herein contains future-looking statements and information within the meaning of Section 27A of the Securities Act of 1933 and Section 21E of the Securities Exchange Act of 1934, including statements regarding expected continual growth of the featured company. Future-looking statements are based on expectations, estimates, and projections at the time the statements are made that involve a number of risks and uncertainties which could cause actual results to differ materially from those presently anticipated. x n wh kmy svlpq t u uvgil ch ufem eksxplpixfxh hhjduyubvkq qxtggmtk juzhxo k kjx gcea From noreply at sourceforge.net Sun Mar 14 20:48:44 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 14 21:37:52 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Sun Mar 14 21:43:37 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 14 21:43:42 2004 Subject: [Patches] [ python-Patches-890203 ] DragonFly BSD support Message-ID: Patches item #890203, was opened at 2004-02-03 22:30 Message generated for change (Comment added) made by jparise You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=890203&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Jon Parise (jparise) Assigned to: Nobody/Anonymous (nobody) Summary: DragonFly BSD support Initial Comment: I spent some time building a simple little patch that gets Python up and running under DragonFly BSD. At the moment, DragonFly (as a platform) is nearly identical to FreeBSD (from which is was forked), so the generated the patch was largely mechanical. I'll continue to make efforts to support Python on DragonFly should things become more complicated. The only known issue is a test case failure for test_fcntl. I don't know whether this failure is unique to DragonFly at this time. I'll investigate that soon. This patch is valid for DragonFly 1.0-CURRENT using either GCC 2.95.4 or GCC 3.3.3 20040126. ---------------------------------------------------------------------- >Comment By: Jon Parise (jparise) Date: 2004-03-14 18:43 Message: Logged In: YES user_id=485579 The test_fcnlt failure has been corrected in the latest patch. ---------------------------------------------------------------------- Comment By: Jon Parise (jparise) Date: 2004-03-04 23:26 Message: Logged In: YES user_id=485579 Updated the patch against the latest Python 2.4a (cvs) code. ---------------------------------------------------------------------- Comment By: Jon Parise (jparise) Date: 2004-02-04 16:00 Message: Logged In: YES user_id=485579 Yes, I'm comfortable with that. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-02-04 14:36 Message: Logged In: YES user_id=21627 Are you willing to act as a maintainer and contact point for the Dragonfly port? ---------------------------------------------------------------------- Comment By: Jon Parise (jparise) Date: 2004-02-03 22:32 Message: Logged In: YES user_id=485579 Also, attached in the generated IN.py Lib/plat-dragonfly1. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=890203&group_id=5470 From noreply at sourceforge.net Sun Mar 14 23:30:51 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 14 23:30:56 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Sun Mar 14 23:55:09 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 14 23:55:12 2004 Subject: [Patches] [ python-Patches-914546 ] Make history recall a-cyclic Message-ID: Patches item #914546, was opened at 2004-03-11 18:00 Message generated for change (Settings changed) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914546&group_id=5470 Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Submitted By: Noam Raphael (noamr) >Assigned to: Kurt B. Kaiser (kbk) Summary: Make history recall a-cyclic Initial Comment: This patch adds an option for idleHistory to be a-cyclic -- holding the Alt-P keys will bring all the commands typed, one by one, and stop at the first one. This, I think, is more intuitive - when history is cyclic, and you press Alt-P continuously, you have no sense of your position, since it's always jumping from end to beginning. That's why I added this option, and that's why I made it a-cyclic by default. The diffs are against idlefork 0.9b1, but work against current idle cvs. Noam ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914546&group_id=5470 From noreply at sourceforge.net Mon Mar 15 03:47:33 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 03:47:39 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 03:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Mon Mar 15 07:17:21 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 07:17:32 2004 Subject: [Patches] [ python-Patches-736962 ] Port tests to unittest (Part 2) Message-ID: Patches item #736962, was opened at 2003-05-13 12:45 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=736962&group_id=5470 Category: Tests Group: None >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Walter D?rwald (doerwalter) Assigned to: Walter D?rwald (doerwalter) Summary: Port tests to unittest (Part 2) Initial Comment: Here are the next test scripts ported to PyUnit: test_winsound and test_array. For test_array many additional tests have been added (code coverage is at 91%) ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2004-03-15 13:17 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/output/test_binascii delete Lib/test/test_binascii.py 1.17 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 21:26 Message: Logged In: YES user_id=80475 Okay, this one is fine. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-19 00:49 Message: Logged In: YES user_id=89016 Here's the next one: test_binascii.py. Code coverage in binascii.c is at 92% (could be improved by enhancing test_binhex.py). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-13 23:46 Message: Logged In: YES user_id=33168 Looked good to me, test_future_pyunit.diff checked in as: * Lib/test/badsyntax_future3.py 1.2 * Lib/test/badsyntax_future4.py 1.2 * Lib/test/badsyntax_future5.py 1.2 * Lib/test/badsyntax_future6.py 1.2 * Lib/test/badsyntax_future7.py 1.2 * Lib/test/badsyntax_future8.py 1.1 * Lib/test/badsyntax_future9.py 1.1 * Lib/test/test_future.py 1.7 * Lib/test/test_future1.py 1.3 * Lib/test/test_future2.py 1.2 * Lib/test/output/test_future 1.4 ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-13 22:47 Message: Logged In: YES user_id=33168 Walter, I didn't get a mail from SF either. After you do a cvs add, you need to use -N to cvs diff. For example, cvs diff -N new_file_added_to_cvs_but_not_committed. If you look carefully, you can see this in the patch on the diff line for each file. I'll take a look at the patch. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-13 21:18 Message: Logged In: YES user_id=89016 Here is a version of test_future ported to PyUnit (test_future_pyunit.diff). If this makes sense to you Neal, please check it in. (BTW, how did you convince CVS to include badsyntax_future8.py and badsyntax_future9.py in the diff? Doing a "cvs add" only results in " Lib/test/badsyntax_future8.py is a new entry, no comparison available") ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-11 13:36 Message: Logged In: YES user_id=89016 md5/weakref patch checked in as: Lib/test/test_weakref.py 1.33 Lib/test/test_md5.py 1.5 Lib/test/output/test_md5 remove ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-11 07:05 Message: Logged In: YES user_id=33168 Improve coverage for test_future too. I'm not sure if test future can be ported to PyUnit. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-12-11 06:52 Message: Logged In: YES user_id=33168 Attached are two tests (in one file, I'm being lazy, sorry) to improve test coverage for md5c.c and _weakref.c. test_md5 was ported to unittest, weakref, just adds two little tests to get coverage up to 100%. If you like, just go ahead and check in. You will need to remove Lib/test/output/test_md5 ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-12-08 12:42 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/list_tests.py 1.1 Lib/test/seq_tests.py 1.1 Lib/test/test_list.py 1.1 Lib/test/test_tuple.py 1.1 Lib/test/test_types.py 1.56 Lib/test/test_userlist.py 1.11 Lib/test/output/test_types 1.3 Next will be test_binascii.py before I continue with test_types.py. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-12-08 00:49 Message: Logged In: YES user_id=80475 Looks good. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-11-27 20:48 Message: Logged In: YES user_id=89016 Here the first part of the test_types port: list and tuple tests have been moved to their own scripts: test_tuple.py and test_list.py. Common tests for tuple, list and UserList are shared (in seq_test.py and list_test.py) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-09-02 03:53 Message: Logged In: YES user_id=80475 Applied as: Lib/test/test_slice.py 1.5. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-09-01 01:52 Message: Logged In: YES user_id=80475 Looks good. I added a couple of minor tests. Revised patch attached. Okay to apply. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-31 21:49 Message: Logged In: YES user_id=89016 Thanks! Here's another one: test_slice.py ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-31 01:04 Message: Logged In: YES user_id=80475 Done. See: Lib/test/test_longexp.py 1.9; previous revision: 1.8 Lib/test/test_pep263.py 1.3; previous revision: 1.2 Lib/test/test_sets.py 1.27; previous revision: 1.26 Lib/test/test_structseq.py 1.5; previous revision: 1.4 Lib/test/output/test_longexp delete; previous revision: 1.3 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-30 19:10 Message: Logged In: YES user_id=80475 Will do! ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-30 19:06 Message: Logged In: YES user_id=89016 Here's test_structse.py converted with a few additional tests. If all three scripts are OK, could you check them in Raymond, as I'll be on vacation for three weeks? ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-08-09 17:47 Message: Logged In: YES user_id=89016 Here are two simple ones: test_pep263 and test_longexp. I can't think of any additional tests to add. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-23 15:38 Message: Logged In: YES user_id=80475 Fixed Walter's review comments and committed as Lib/test/test_compile.py 1.19 ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-23 13:49 Message: Logged In: YES user_id=89016 Are you sure, that the code path is the same in the new test_argument_handling() as in the old test? I.e. is "eval('lambda a,a: 0')" the same as "exec 'def f(a, a): pass'"? The print statement in test_float_literals should be changed to a comment. Should the imports in test_unary_minus() be moved to the start of the script? Otherwise the test looks (and runs) OK. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-20 21:26 Message: Logged In: YES user_id=80475 Here's one for you: test_compile.py is ready. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-19 11:46 Message: Logged In: YES user_id=89016 Maybe test_types.py should be split into several scripts: test_dict, test_tuple, test_list, test_int, test_long etc. Some of them already exist (like test_long), some don't. The constructor tests from test_builtin should probably be moved to the new test scripts as well. Furthermore we should try to share as much testing functionality as possible (e.g. between test_int and test_long, or between test_list and test_userlist) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-18 21:48 Message: Logged In: YES user_id=80475 Great! Can I suggest that test_types.py be next. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-18 16:27 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/test_builtin.py 1.21 Lib/test/test_complex.py 1.10 (I've moved the constructor tests from test_builtin to test_complex) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-18 01:49 Message: Logged In: YES user_id=80475 I added a few tests. If they are fine with you, go ahead and commit. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-17 23:36 Message: Logged In: YES user_id=89016 Here's the next one: text_complex.py. There one test that's currently commented out, because it crashes Python on Alpha (see http://www.python.org/sf/756093. The old test scripts states that tests for the constructor are in test_builtin, but I've added many tests to this script, so this is no longer true. We could move the tests to test_builtin, but IMHO that doesn't make sense, we'd better move the rest of the constructor tests from test_builtin to test_complex. I'd like to have a version of assertAlmostEqual() in unittest.py that can cope with complex numbers, but this would have to be coordinated with the standalone version of PyUnit (and it would probably have to wait until the 2.4 cycle starts) (I noticed that there is no assertAlmostEqual in the code on pyunit.sf.net anyway.) ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-17 14:06 Message: Logged In: YES user_id=89016 I'd like to keep this patch open, as it is an ongoing task (the next test scripts to be converted will be test_complex and then maybe test_marshal) ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-06-16 23:55 Message: Logged In: YES user_id=357491 OK, all tests pass cleanly. Applied as revision 1.6. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 21:51 Message: Logged In: YES user_id=89016 The joys of unittesting: Breaking code to make it better! ;) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 21:41 Message: Logged In: YES user_id=80475 Okay, it runs fine here. Once Brett confirms that it runs on the Mac, go ahead and load it. P.S. Your improved test_mimetools.py helped detect a latent error in mimetools.py when it was run under Windows. Tim made the fix this weekend. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 21:33 Message: Logged In: YES user_id=89016 Strange, I didn't get this failure here, but I got it on my laptop at home. I've removed the comparison with the getatime() value from test_time(). I hope this fixes it. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 21:09 Message: Logged In: YES user_id=80475 Walter, there is one failure left: ====================================== ================================ FAIL: test_time (__main__.PosixPathTest) ------------------------------------------------------------------- --- Traceback (most recent call last): File "test_posixpath.py", line 125, in test_time self.assert_( File "C:\PY23\lib\unittest.py", line 268, in failUnless if not expr: raise self.failureException, msg AssertionError Brett, after Walter revises the patch, just load the patch and make sure the test runs on the Mac. Between the three of us, we can validate the suite on three different platforms. Cheers. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-06-16 20:20 Message: Logged In: YES user_id=357491 Just wanting to work me like a dog, huh, Raymond? =) And to clarify for my and Walter's benefit, when you say guards, you mean that the tests don't crap out and say they failed on Windows, right? I thought posixpath was not meant to work under Windows. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 20:09 Message: Logged In: YES user_id=89016 I didn't realize that test_posixpath must work on Windows too. Here's a new version. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 18:04 Message: Logged In: YES user_id=80475 The previous comment applied to another patch. It should have said: Assigning to Brett to make sure the patch runs on the Mac. Don't accept this one until it has guards that allow the tests to run on Windows. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 17:59 Message: Logged In: YES user_id=80475 Assigning to Brett to give experience doing a detail review on this type of change. * examine every line of the diff and consider whether there is any semantic change (exceptions raised, etc). * apply the diff and run the test suite * in the interactive mode, call-up each function and make sure it behaves as expected (this is necessary because the test coverage is very low). * verify that the whitespace has been cleaned up. * look for missing changes (such as use of +=) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-16 17:44 Message: Logged In: YES user_id=80475 The test file now has dependencies that do not apply to windows. The failure messages are attached. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-06-16 14:48 Message: Logged In: YES user_id=89016 Here's the next one: test_posixpath.py with many additional tests. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-22 19:33 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/output/test_mimetools delete Lib/test/test_mimetools.py 1.4 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-22 18:18 Message: Logged In: YES user_id=80475 test_mimetools.py is ready. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-22 17:05 Message: Logged In: YES user_id=89016 I've attached a third version of test_mimetools.py that does some checks for the mimetools.Message class. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-21 15:04 Message: Logged In: YES user_id=80475 Attaching a slightly modified test_mimetools which covers more encodings and has a stronger set test. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-19 01:46 Message: Logged In: YES user_id=89016 Agreed, this is too much magic for too little gain. Back to business: Here is test_mimetools ported to PyUnit. Tests for mimetools.Message are still missing. If you can think of any tests please add them. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-18 05:18 Message: Logged In: YES user_id=80475 Get the module with sys.modules: tests = test_support.findtestclasses(sys.modules [__name__]) test_support.unittest(*tests) Yeah, the inheritance thing is a problem. I was trying to avoid having to modify unittest.TestCase to have a metaclass. The control of the module is kept in a separate SF project and one of its goals is to be backward compatible through 1.5.2 (meaning no metaclasses). A possible workaround is to define a modified testcase in test_support so that people don't import unittest directly anymore: test_support.py ------------------------- import unittest class SmartTestCase(unittest.TestCase): __metaclass__ = autotracktests pass test_sets.py ------------------ class TestBasicOps(test_support.SmartTestCase): run = False . . . class TestBasicOpsEmpty(TestBasicOps): def setUp(self): . . . Still, this is starting to seem a bit magical and tricky. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-18 04:52 Message: Logged In: YES user_id=89016 But how do I pass the module object from inside the module? And skipping abstract classes seems to be more work in this version: If skipping is done via a class attribute, derived classes have to explicitely reset this flag because of interitance. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-18 03:59 Message: Logged In: YES user_id=80475 Good call. Instead of using metaclasses, perhaps add a module introspector function to test_support: def findtestclasses(mod): tests = [] for elem in dir(mod): member = getattr(mod, elem) if type(member) != type: continue if issubclass(member, unittest.TestCase): tests.append(member) return tests ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-18 03:45 Message: Logged In: YES user_id=89016 But this can be solved with a special non-inheritable class attribute: class BaseTest(unittest.TestCase): run = False Then the metaclass can do the following: def __new__(cls, name, bases, dict): if "run" not in dict: dict["run"] = True cls = type.__new__(cls, name, bases, dict) if cls.run: tests.append(cls) return cls ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-18 03:04 Message: Logged In: YES user_id=80475 I don't think metaclasses or module introspection would help whenever there are classes that derive from TestCase but are not meant to be run directly (their subclasses have the setup/teardown/or class data). test_sets.py has examples of that kind of thing. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2003-05-18 02:50 Message: Logged In: YES user_id=89016 Checked in as: Lib/test/test_array.py 1.20 Lib/test/test_winsound.py 1.5 Lib/test/output/test_winsound delete > The approach of using tests.append() is elegant and > makes it easier to verify that no tests are being omitted. The most elegant approach would probably be a metaclass that collects all TestCase subclasses that get defined. Classes that only serve as a base class could be skipped by specifying a certain class attribute. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-18 01:35 Message: Logged In: YES user_id=80475 The approach of using tests.append() is elegant and makes it easier to verify that no tests are being omitted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=736962&group_id=5470 From noreply at sourceforge.net Mon Mar 15 15:51:50 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 16:02:15 2004 Subject: [Patches] [ python-Patches-916874 ] fix for bug #857297 (tarfile and hardlinks) Message-ID: Patches item #916874, was opened at 2004-03-15 21:51 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916874&group_id=5470 Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: fix for bug #857297 (tarfile and hardlinks) Initial Comment: Included is a patch that fixes the extraction of hardlinks in tarfile.py (bug #857297) and a testcase. Until now tarfile.py extracted hardlinks the same way as symlinks which worked under certain circumstances: if the hardlink source is accessible from the current working directory everything is fine, but if you choose to extract it to a different location it won't work. That particular code in tarfile.py dates back to my good old windoze days where my understanding of unix filesystem basics was not as profound as it is today :-) The testcase probably needs some attention regarding portability. (On platforms that do not have os.link() it is skipped completely.) Trying to extract a hardlink and testing for its existence afterwards is no option here, because it doesn't necessarily need to be tarfile.py's fault if this doesn't work. There are many different conditions involved, especially the filesystem the test is made on. My current solution is that the testcase tries to create a link and if it gets an ENOENT (which means the link's source file cannot be found) it fails. All other EnvironmentErrors are ignored. Interestingly this testcase even works on a FAT32 filesystem (on my Linux box) - if the link's source cannot be found there is an ENOENT (and the test fails as it should), and if it can be found there is an EPERM which is ignored. IMO all this is a bit too much black magic and maybe the testcase introduces more problems than it actually solves, so feel free to throw it away. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916874&group_id=5470 From noreply at sourceforge.net Mon Mar 15 20:15:21 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 20:15:27 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 20:15 Message: Logged In: YES user_id=80475 Added a revised patch that can save even more work. If the previously freed dictionary has mp->mp_fill==0, then the memset() initialization can be skipped. Instrumenting this (against several different apps and the test suite) shows that 4 out of 5 mallocs are saved and the 1 in 10 memsets are saved. The counts fall off by about a quarter if the size of the freelist is cut in half (suggesting that 60 to 100 freedicts is plenty). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 03:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Mon Mar 15 21:57:53 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 21:58:03 2004 Subject: [Patches] [ python-Patches-916874 ] fix for bug #857297 (tarfile and hardlinks) Message-ID: Patches item #916874, was opened at 2004-03-15 15:51 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916874&group_id=5470 Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: fix for bug #857297 (tarfile and hardlinks) Initial Comment: Included is a patch that fixes the extraction of hardlinks in tarfile.py (bug #857297) and a testcase. Until now tarfile.py extracted hardlinks the same way as symlinks which worked under certain circumstances: if the hardlink source is accessible from the current working directory everything is fine, but if you choose to extract it to a different location it won't work. That particular code in tarfile.py dates back to my good old windoze days where my understanding of unix filesystem basics was not as profound as it is today :-) The testcase probably needs some attention regarding portability. (On platforms that do not have os.link() it is skipped completely.) Trying to extract a hardlink and testing for its existence afterwards is no option here, because it doesn't necessarily need to be tarfile.py's fault if this doesn't work. There are many different conditions involved, especially the filesystem the test is made on. My current solution is that the testcase tries to create a link and if it gets an ENOENT (which means the link's source file cannot be found) it fails. All other EnvironmentErrors are ignored. Interestingly this testcase even works on a FAT32 filesystem (on my Linux box) - if the link's source cannot be found there is an ENOENT (and the test fails as it should), and if it can be found there is an EPERM which is ignored. IMO all this is a bit too much black magic and maybe the testcase introduces more problems than it actually solves, so feel free to throw it away. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 21:57 Message: Logged In: YES user_id=33168 Lars, I just wanted to say thanks for this patch and the other. I hope to get around to them soon. But it may have to wait until after pycon. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916874&group_id=5470 From noreply at sourceforge.net Mon Mar 15 22:05:11 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 22:05:47 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:05 Message: Logged In: YES user_id=33168 Looking at the patch now, but I noticed a seemingly useless comparison in characterize around line 1328 (not related to your work): if (cmp > 0 || i > a->ma_mask || a->ma_table[i].me_value == NULL) I see no way for i > a->ma_mask based off the for loop (i <= a->ma_mask). Am I missing something? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 20:15 Message: Logged In: YES user_id=80475 Added a revised patch that can save even more work. If the previously freed dictionary has mp->mp_fill==0, then the memset() initialization can be skipped. Instrumenting this (against several different apps and the test suite) shows that 4 out of 5 mallocs are saved and the 1 in 10 memsets are saved. The counts fall off by about a quarter if the size of the freelist is cut in half (suggesting that 60 to 100 freedicts is plenty). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 03:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Mon Mar 15 22:12:21 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 22:12:28 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-15 22:12 Message: Logged In: YES user_id=31435 Neal, study the comment following that code. Sick code could have reduced the size of the dict as a side effect of calling PyObject_RichCompareBool(); that would make ma_mask smaller than it was at the start of the loop. Nothing about a dict can be assumed invariant across anything that may release the GIL or call back into Python (except for the address of the dict object). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:05 Message: Logged In: YES user_id=33168 Looking at the patch now, but I noticed a seemingly useless comparison in characterize around line 1328 (not related to your work): if (cmp > 0 || i > a->ma_mask || a->ma_table[i].me_value == NULL) I see no way for i > a->ma_mask based off the for loop (i <= a->ma_mask). Am I missing something? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 20:15 Message: Logged In: YES user_id=80475 Added a revised patch that can save even more work. If the previously freed dictionary has mp->mp_fill==0, then the memset() initialization can be skipped. Instrumenting this (against several different apps and the test suite) shows that 4 out of 5 mallocs are saved and the 1 in 10 memsets are saved. The counts fall off by about a quarter if the size of the freelist is cut in half (suggesting that 60 to 100 freedicts is plenty). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 03:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Mon Mar 15 22:15:55 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 22:16:03 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Neal Norwitz (nnorwitz) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:15 Message: Logged In: YES user_id=33168 Thanks Tim. I saw it late last night and didn't re-read today. Too much PSF work, not enough coding. :-( ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-15 22:12 Message: Logged In: YES user_id=31435 Neal, study the comment following that code. Sick code could have reduced the size of the dict as a side effect of calling PyObject_RichCompareBool(); that would make ma_mask smaller than it was at the start of the loop. Nothing about a dict can be assumed invariant across anything that may release the GIL or call back into Python (except for the address of the dict object). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:05 Message: Logged In: YES user_id=33168 Looking at the patch now, but I noticed a seemingly useless comparison in characterize around line 1328 (not related to your work): if (cmp > 0 || i > a->ma_mask || a->ma_table[i].me_value == NULL) I see no way for i > a->ma_mask based off the for loop (i <= a->ma_mask). Am I missing something? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 20:15 Message: Logged In: YES user_id=80475 Added a revised patch that can save even more work. If the previously freed dictionary has mp->mp_fill==0, then the memset() initialization can be skipped. Instrumenting this (against several different apps and the test suite) shows that 4 out of 5 mallocs are saved and the 1 in 10 memsets are saved. The counts fall off by about a quarter if the size of the freelist is cut in half (suggesting that 60 to 100 freedicts is plenty). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 03:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Mon Mar 15 23:16:16 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Mon Mar 15 23:16:25 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Hye-Shik Chang (perky) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Tue Mar 16 00:33:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 16 00:33:27 2004 Subject: [Patches] [ python-Patches-917095 ] dict type concat function Message-ID: Patches item #917095, was opened at 2004-03-16 05:33 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: troy melhase (troy_melhase) Assigned to: Nobody/Anonymous (nobody) Summary: dict type concat function Initial Comment: Adds a function to dictobject.c that allows python expressions like: >>> d = {2:3} >>> d += {4:5} >>> d {2: 3, 4: 5} and like: >>> d = {2:3} >>> e = d + {6:7} >>> e {2: 3, 6: 7} A few points: * I don't know much C, and this patch is probably implemented much more appropriately by someone who does * I don't know if there's a good reason that the dict type doesn't already supply this; if that's the case, I'd be interested to know why not * Lib/test/test_builtin.py fails (as it should) after applying this patch Please advise, and thanks! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 From noreply at sourceforge.net Tue Mar 16 06:08:10 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 16 06:08:15 2004 Subject: [Patches] [ python-Patches-917095 ] dict type concat function Message-ID: Patches item #917095, was opened at 2004-03-16 00:33 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: troy melhase (troy_melhase) Assigned to: Nobody/Anonymous (nobody) Summary: dict type concat function Initial Comment: Adds a function to dictobject.c that allows python expressions like: >>> d = {2:3} >>> d += {4:5} >>> d {2: 3, 4: 5} and like: >>> d = {2:3} >>> e = d + {6:7} >>> e {2: 3, 6: 7} A few points: * I don't know much C, and this patch is probably implemented much more appropriately by someone who does * I don't know if there's a good reason that the dict type doesn't already supply this; if that's the case, I'd be interested to know why not * Lib/test/test_builtin.py fails (as it should) after applying this patch Please advise, and thanks! ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-16 06:08 Message: Logged In: YES user_id=80475 I recommend against this for several reasons: 1) adds new syntax but not new functionality 2) concept doesn't usefully extend to - and * 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() 4) no use cases showing the + operator to be more expressive 5) possibly confusing given the | operator is used for sets 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. If for some reason this gets approved, please assign back to me for implementation, documentation, and unittests. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 From aikokucht at rpcm.com Tue Mar 16 11:34:14 2004 From: aikokucht at rpcm.com (Mohamed Willis) Date: Tue Mar 16 08:26:37 2004 Subject: [Patches] Your friend set you up Message-ID: Interested in a connection that was set up by a mutual friend? Click here to accept the invitation: http://needtofindlovechat.com/confirm/?oc=50798655 The FREE dating web site CREATED BY WOMEN Click here if you do not wish to be invited again: http://needtofindlovechat.com/remove/?oc=50791111 Click here if you do not wish to be invited again: http://fightnottobesingle.com/remove/?oc=xxxxyyyy nonce brine pitt credent propulsion hunch cosgrove handgun bondsmen sanguine truncate propelling pellagra feeney geese infima 4 privilege petrifaction jessie market capture landmark depose freon parishioner sling carpathia cantilever niece aliquot callus modish nucleate goat bryn lopsided edmonton hunter empire crafty byronic coven basidiomycetes chive beirut brant tubule priscilla 1 From noreply at sourceforge.net Tue Mar 16 11:24:52 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 16 11:25:13 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by jiwon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Hye-Shik Chang (perky) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From rcjbw66 at ya.com Tue Mar 16 09:53:56 2004 From: rcjbw66 at ya.com (Elias Burr) Date: Tue Mar 16 11:59:28 2004 Subject: [Patches] dysplasia Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040316/64cfee56/attachment.html From noreply at sourceforge.net Tue Mar 16 14:19:37 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 16 14:19:42 2004 Subject: [Patches] [ python-Patches-873305 ] list.__setitem__(slice) behavior Message-ID: Patches item #873305, was opened at 2004-01-08 15:46 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Brandmeyer (jbrandmeyer) Assigned to: Nobody/Anonymous (nobody) Summary: list.__setitem__(slice) behavior Initial Comment: Consider the following: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0] = [5, 6, 7] Currently the result is that x == [0,1,2,3,5,6,7,4]. However, I believe that calling setitem with an empty slice should be a no-op, rather than performing an insertion starting at the 'begin' index of the slice. The following patch to Objects/listobject.c makes this change. Thanks, Jonathan Brandmeyer ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2004-03-16 14:19 Message: Logged In: YES user_id=593130 Empty slice 'replacement' as insertion is an intentional feature. It actually defines indexed insertion: Lib REf 2.3.6.4 Mutable Sequence Types says "s.insert(i, x) same as s[i:i] = [x] (5) " .insert was recently changed to make this exactly true: "(5) When a negative index is passed as the first parameter to the insert() method, the list length is added, as for slice indices. If it is still negative, it is truncated to zero, as for slice indices. Changed in version 2.3: Previously, all negative indices were truncated to zero." It is also a feature (intentional, I presume) of slices that they always work (unlike indices). The current behavior is what I expect. To be clearer, "s[i:j] = t slice of s from i to j is replaced by t " in the table should grow a footnote: "If j < i, j becomes i." (or "is replaced by") This is my suggested resolution of this item (but I am not a Latexer). As for extended slice assignment, "should mean exactly the same thing" is simply wrong as a general statement. The manual says "s[i:j:k]=t the elements of s[i:j:k] are replaced by those of t (1)" and the footnote says "(1) t must have the same length as the slice it is replacing" which is quite different simple slice replacement. The error message has nothing to do with the 0 length of the target other than that it is different from the source. s[0:1:1] = [5,6] is also invalid even though s[0:1] = [5,6] works (given len(s) >= 1). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 01:14 Message: Logged In: YES user_id=80475 IMO, the risks of changing this outweight any possible benefits. I would be interested to see whether others would rally for or against this one. ---------------------------------------------------------------------- Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:54 Message: Logged In: YES user_id=676765 Here is a snippet from a conversation that occured on the python-c++ list in January: > So now the question is, > what is the result of replacing an empty slice in a container with a > non-empty sequence? If the container supports insertion, inserting the > sequence seems logical enough to me. So how do you define the point to perform the insertion? If the user asks to replace every element that is greater than or equal to the fourth and less than the first element, where do you place the new sequence? The current action for a built-in list is to range-limit the stop point to be not less than the start point, but I think that is just for safety rather than an API decision, and that it is wrong. ---------------------------------------------------------------------- Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:42 Message: Logged In: YES user_id=676765 Consider this alternate example which should mean exactly the same thing, but doesn't: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0:1] = [5, 6, 7] Traceback (most recent call last): File "", line 1, in ? ValueError: attempt to assign sequence of size 3 to extended slice of size 0 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:16 Message: Logged In: YES user_id=80475 I believe we are stuck with this one. Assigning to an empty slice like a[2:2]=list('hotdog') is a valid and useful. The weird part is that the rightmost index gets adjusted to make the most sense given the leftmost index. In your example, the offerred indices of -1 and 0 would seem to indicate a programming error. However, this autoadjusting is a long standing feature and occassionally useful in its own right. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 From noreply at sourceforge.net Tue Mar 16 14:22:41 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 16 14:22:45 2004 Subject: [Patches] [ python-Patches-917095 ] dict type concat function Message-ID: Patches item #917095, was opened at 2004-03-16 05:33 Message generated for change (Comment added) made by troy_melhase You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: troy melhase (troy_melhase) Assigned to: Nobody/Anonymous (nobody) Summary: dict type concat function Initial Comment: Adds a function to dictobject.c that allows python expressions like: >>> d = {2:3} >>> d += {4:5} >>> d {2: 3, 4: 5} and like: >>> d = {2:3} >>> e = d + {6:7} >>> e {2: 3, 6: 7} A few points: * I don't know much C, and this patch is probably implemented much more appropriately by someone who does * I don't know if there's a good reason that the dict type doesn't already supply this; if that's the case, I'd be interested to know why not * Lib/test/test_builtin.py fails (as it should) after applying this patch Please advise, and thanks! ---------------------------------------------------------------------- >Comment By: troy melhase (troy_melhase) Date: 2004-03-16 19:22 Message: Logged In: YES user_id=548370 1) adds new syntax but not new functionality 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() The same is true of list.extend and list.append. After years of writing python code, I still forget that I can't add dictionaries to one another, which for me at least, violates the principle of least surprise. The patch doesn't provide the same behavior as the update method; when adding two dicts, a third dict is returned. Only in the case of augmented assignment is the update method similar. Moreover, every other basic container-ish type supports addition of the same type (or similar type). It seems to me that the dict type is missing a feature, not the other way around. 2) concept doesn't usefully extend to - and * I could see it extending to -, but you're right that it doesn't extend to *. 5) possibly confusing given the | operator is used for sets I don't understand this point, could you elaborate? 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. But unlike list addition, dict addition is commutative: >>> d = {2:3} >>> e = {4:5} >>> d + e == e + d True Thanks for looking; could you advise again? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-16 11:08 Message: Logged In: YES user_id=80475 I recommend against this for several reasons: 1) adds new syntax but not new functionality 2) concept doesn't usefully extend to - and * 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() 4) no use cases showing the + operator to be more expressive 5) possibly confusing given the | operator is used for sets 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. If for some reason this gets approved, please assign back to me for implementation, documentation, and unittests. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 From noreply at sourceforge.net Tue Mar 16 14:36:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 16 14:36:08 2004 Subject: [Patches] [ python-Patches-873305 ] list.__setitem__(slice) behavior Message-ID: Patches item #873305, was opened at 2004-01-08 15:46 Message generated for change (Comment added) made by jbrandmeyer You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Jonathan Brandmeyer (jbrandmeyer) Assigned to: Nobody/Anonymous (nobody) Summary: list.__setitem__(slice) behavior Initial Comment: Consider the following: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0] = [5, 6, 7] Currently the result is that x == [0,1,2,3,5,6,7,4]. However, I believe that calling setitem with an empty slice should be a no-op, rather than performing an insertion starting at the 'begin' index of the slice. The following patch to Objects/listobject.c makes this change. Thanks, Jonathan Brandmeyer ---------------------------------------------------------------------- >Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-16 14:36 Message: Logged In: YES user_id=676765 Very well. Thanks for the clarification. ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2004-03-16 14:19 Message: Logged In: YES user_id=593130 Empty slice 'replacement' as insertion is an intentional feature. It actually defines indexed insertion: Lib REf 2.3.6.4 Mutable Sequence Types says "s.insert(i, x) same as s[i:i] = [x] (5) " .insert was recently changed to make this exactly true: "(5) When a negative index is passed as the first parameter to the insert() method, the list length is added, as for slice indices. If it is still negative, it is truncated to zero, as for slice indices. Changed in version 2.3: Previously, all negative indices were truncated to zero." It is also a feature (intentional, I presume) of slices that they always work (unlike indices). The current behavior is what I expect. To be clearer, "s[i:j] = t slice of s from i to j is replaced by t " in the table should grow a footnote: "If j < i, j becomes i." (or "is replaced by") This is my suggested resolution of this item (but I am not a Latexer). As for extended slice assignment, "should mean exactly the same thing" is simply wrong as a general statement. The manual says "s[i:j:k]=t the elements of s[i:j:k] are replaced by those of t (1)" and the footnote says "(1) t must have the same length as the slice it is replacing" which is quite different simple slice replacement. The error message has nothing to do with the 0 length of the target other than that it is different from the source. s[0:1:1] = [5,6] is also invalid even though s[0:1] = [5,6] works (given len(s) >= 1). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 01:14 Message: Logged In: YES user_id=80475 IMO, the risks of changing this outweight any possible benefits. I would be interested to see whether others would rally for or against this one. ---------------------------------------------------------------------- Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:54 Message: Logged In: YES user_id=676765 Here is a snippet from a conversation that occured on the python-c++ list in January: > So now the question is, > what is the result of replacing an empty slice in a container with a > non-empty sequence? If the container supports insertion, inserting the > sequence seems logical enough to me. So how do you define the point to perform the insertion? If the user asks to replace every element that is greater than or equal to the fourth and less than the first element, where do you place the new sequence? The current action for a built-in list is to range-limit the stop point to be not less than the start point, but I think that is just for safety rather than an API decision, and that it is wrong. ---------------------------------------------------------------------- Comment By: Jonathan Brandmeyer (jbrandmeyer) Date: 2004-03-13 16:42 Message: Logged In: YES user_id=676765 Consider this alternate example which should mean exactly the same thing, but doesn't: >>> x = [0, 1, 2, 3, 4] >>> x[-1:0:1] = [5, 6, 7] Traceback (most recent call last): File "", line 1, in ? ValueError: attempt to assign sequence of size 3 to extended slice of size 0 ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:16 Message: Logged In: YES user_id=80475 I believe we are stuck with this one. Assigning to an empty slice like a[2:2]=list('hotdog') is a valid and useful. The weird part is that the rightmost index gets adjusted to make the most sense given the leftmost index. In your example, the offerred indices of -1 and 0 would seem to indicate a programming error. However, this autoadjusting is a long standing feature and occassionally useful in its own right. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873305&group_id=5470 From noreply at sourceforge.net Tue Mar 16 14:35:00 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 16 16:31:08 2004 Subject: [Patches] [ python-Patches-917095 ] dict type concat function Message-ID: Patches item #917095, was opened at 2004-03-16 00:33 Message generated for change (Comment added) made by tjreedy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: troy melhase (troy_melhase) Assigned to: Nobody/Anonymous (nobody) Summary: dict type concat function Initial Comment: Adds a function to dictobject.c that allows python expressions like: >>> d = {2:3} >>> d += {4:5} >>> d {2: 3, 4: 5} and like: >>> d = {2:3} >>> e = d + {6:7} >>> e {2: 3, 6: 7} A few points: * I don't know much C, and this patch is probably implemented much more appropriately by someone who does * I don't know if there's a good reason that the dict type doesn't already supply this; if that's the case, I'd be interested to know why not * Lib/test/test_builtin.py fails (as it should) after applying this patch Please advise, and thanks! ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2004-03-16 14:34 Message: Logged In: YES user_id=593130 I am usually big on orthogonality, but it never occurred to me that dicts should be addible, although sets are. What should be the result of {1:2} + {1:3}? If not something like {1:set (2,3)}, the operator is *not* commutative. The reference to sets and | is that | is used, I believe, for set union (addition) (and & for intersection). So, if there were to be a dict union operator, the symbol should be | rather than +. ---------------------------------------------------------------------- Comment By: troy melhase (troy_melhase) Date: 2004-03-16 14:22 Message: Logged In: YES user_id=548370 1) adds new syntax but not new functionality 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() The same is true of list.extend and list.append. After years of writing python code, I still forget that I can't add dictionaries to one another, which for me at least, violates the principle of least surprise. The patch doesn't provide the same behavior as the update method; when adding two dicts, a third dict is returned. Only in the case of augmented assignment is the update method similar. Moreover, every other basic container-ish type supports addition of the same type (or similar type). It seems to me that the dict type is missing a feature, not the other way around. 2) concept doesn't usefully extend to - and * I could see it extending to -, but you're right that it doesn't extend to *. 5) possibly confusing given the | operator is used for sets I don't understand this point, could you elaborate? 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. But unlike list addition, dict addition is commutative: >>> d = {2:3} >>> e = {4:5} >>> d + e == e + d True Thanks for looking; could you advise again? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-16 06:08 Message: Logged In: YES user_id=80475 I recommend against this for several reasons: 1) adds new syntax but not new functionality 2) concept doesn't usefully extend to - and * 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() 4) no use cases showing the + operator to be more expressive 5) possibly confusing given the | operator is used for sets 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. If for some reason this gets approved, please assign back to me for implementation, documentation, and unittests. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 From noreply at sourceforge.net Wed Mar 17 00:37:11 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 00:37:20 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Hye-Shik Chang (perky) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From lobeld at netster.com Wed Mar 17 03:55:02 2004 From: lobeld at netster.com (Tomas Tate) Date: Wed Mar 17 00:56:07 2004 Subject: [Patches] you could be john holmes Message-ID: Add another few inches to your meat and then REALLY lay the PIPE to her http://ellipsoid.shzze.com/vp5 Click here if you do not wish to be invited again: http://needtofindlovechat.com/remove/?oc=50791111 take off http://findingunity.com/remove/?oc=xxxxyyyy deface chairwoman sack alizarin starkey pinehurst aluminate cecropia mortgagor sidney doria arturo beardsley bolshevism brick desuetude heavyweight correspond 9 efficacious flashlight salmon doctrinaire catnip official caravan escherichia crutch posthumous 3 From noreply at sourceforge.net Wed Mar 17 01:21:02 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 01:21:08 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 07:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) >Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 01:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 00:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-16 11:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-15 23:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-13 21:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-23 11:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 08:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 05:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 01:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 06:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 04:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 07:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 07:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 06:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 03:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 00:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-29 22:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 06:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 02:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-26 10:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 01:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-25 18:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-21 13:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 09:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 06:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-11 22:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 20:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 00:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-07 23:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 07:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Wed Mar 17 01:25:49 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 01:25:55 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) >Assigned to: Hye-Shik Chang (perky) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 01:25 Message: Logged In: YES user_id=80475 Hye-Shik, can you give this a second review? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:15 Message: Logged In: YES user_id=33168 Thanks Tim. I saw it late last night and didn't re-read today. Too much PSF work, not enough coding. :-( ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-15 22:12 Message: Logged In: YES user_id=31435 Neal, study the comment following that code. Sick code could have reduced the size of the dict as a side effect of calling PyObject_RichCompareBool(); that would make ma_mask smaller than it was at the start of the loop. Nothing about a dict can be assumed invariant across anything that may release the GIL or call back into Python (except for the address of the dict object). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:05 Message: Logged In: YES user_id=33168 Looking at the patch now, but I noticed a seemingly useless comparison in characterize around line 1328 (not related to your work): if (cmp > 0 || i > a->ma_mask || a->ma_table[i].me_value == NULL) I see no way for i > a->ma_mask based off the for loop (i <= a->ma_mask). Am I missing something? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 20:15 Message: Logged In: YES user_id=80475 Added a revised patch that can save even more work. If the previously freed dictionary has mp->mp_fill==0, then the memset() initialization can be skipped. Instrumenting this (against several different apps and the test suite) shows that 4 out of 5 mallocs are saved and the 1 in 10 memsets are saved. The counts fall off by about a quarter if the size of the freelist is cut in half (suggesting that 60 to 100 freedicts is plenty). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 03:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Wed Mar 17 01:28:53 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 01:29:01 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) >Assigned to: Hye-Shik Chang (perky) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Wed Mar 17 01:39:45 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 01:39:56 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Settings changed) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) >Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Wed Mar 17 02:15:34 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 02:15:44 2004 Subject: [Patches] [ python-Patches-909005 ] asyncore fixes and improvements Message-ID: Patches item #909005, was opened at 2004-03-03 16:07 Message generated for change (Comment added) made by klimkin You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Alexey Klimkin (klimkin) Assigned to: A.M. Kuchling (akuchling) Summary: asyncore fixes and improvements Initial Comment: Minor: * 0/1 for boolean values replaced with False/True. * (887279) Added handling of POLLPRI as POLLIN. POLLERR, POLLHUP, POLLNVAL are handled as exception event. handle_expt_event gets recent error from self.socket object and raises socket.error. * Default readable()/writable() returns False. * Added "map" parameter for file_dispatcher. * file_wrapper: removed "return" in close(), recv/read and send/write swapped because of their nature. * mac code for writable() removed. Manual for accept() on mac is similar to the one on linux. * Repeating exception changed from "raise socket.error, why" to raise. * Added connected/accepting/addr reset on close(). Initialization of variables moved to __init__. * close_all() now calls close for dispatcher object, EBADF treated as already closed socket/file. * Added channel id to "unhandled..." messages. Bugs: * Fixed bug (654766,889153): client never gets connected, nor errored. Connecting client gets writable event from select(), however, some client may want always be non writable. Such client may never get connected. The fix adds _readable() - always True for accepting and always False for connecting socket; and _writable() - always False for accepting and always True for connecting socket. This implies, that listening dispatcher's readable() and writable() will never be called. ("man accept" and "man connect" for non-blocking sockets). * Fixed bug: error handling after accept(). It's said, that accept can return EWOULDBLOCK even for readable socket. This mean, that even after handle_accept(), dispatcher's accept() still raise EWOULDBLOCK. New code does accept() itself and stores accepted socket in self.__pending_accept. If there was socket.error, it's treated as EWOULDBLOCK. dispatcher's accept returns self.__pending_accept and resets it to None. Features: * Added pending_read() and pending_write(). The functions helps to use dispatcher over non socket objects with buffering capabilities. In original dispatcher, if socket makes buffered read and some data is in buffer, entering asyncore.poll() doesn't finishes, since there is no data in real file/socket. This feature allow to use SSL socket, since the socket reads data by 16k chunks. ---------------------------------------------------------------------- >Comment By: Alexey Klimkin (klimkin) Date: 2004-03-17 10:15 Message: Logged In: YES user_id=410460 Sorry, unfortunately I have lost old patch file. I have atached new one. In addition to fixes, listed above, the patch includes: 1. Fix for operating on uninitialized socket. self.socket now initializes with _closed_socket(), so any operation throws EBADF. 2. Added class idispatcher - base class for dispatcher. The purpose of this class is to allow simple replacement of media(dispatcher interface) in classes, derived from dispatcher class. This is based on 'object'. I have also attached asynchat.diff - example for new-style dispatcher. Old asynchat works as well. ---------------------------------------------------------------------- Comment By: Wummel (calvin) Date: 2004-03-11 18:49 Message: Logged In: YES user_id=9205 There is no file attached! You have to click on the checkbox next to the upload filename. This is a Sourceforge annoyance :( ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=909005&group_id=5470 From noreply at sourceforge.net Wed Mar 17 02:33:59 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 02:34:03 2004 Subject: [Patches] [ python-Patches-917095 ] dict type concat function Message-ID: Patches item #917095, was opened at 2004-03-16 05:33 Message generated for change (Comment added) made by troy_melhase You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed Resolution: None Priority: 5 Submitted By: troy melhase (troy_melhase) Assigned to: Nobody/Anonymous (nobody) Summary: dict type concat function Initial Comment: Adds a function to dictobject.c that allows python expressions like: >>> d = {2:3} >>> d += {4:5} >>> d {2: 3, 4: 5} and like: >>> d = {2:3} >>> e = d + {6:7} >>> e {2: 3, 6: 7} A few points: * I don't know much C, and this patch is probably implemented much more appropriately by someone who does * I don't know if there's a good reason that the dict type doesn't already supply this; if that's the case, I'd be interested to know why not * Lib/test/test_builtin.py fails (as it should) after applying this patch Please advise, and thanks! ---------------------------------------------------------------------- >Comment By: troy melhase (troy_melhase) Date: 2004-03-17 07:33 Message: Logged In: YES user_id=548370 Doh. You're right about it being not commutative. I hadn't considered the example you provided. I agree that the | operator makes more sense. I'll close this patch until I'm up to implementing that. Thanks, Terry, Raymond! ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2004-03-16 19:34 Message: Logged In: YES user_id=593130 I am usually big on orthogonality, but it never occurred to me that dicts should be addible, although sets are. What should be the result of {1:2} + {1:3}? If not something like {1:set (2,3)}, the operator is *not* commutative. The reference to sets and | is that | is used, I believe, for set union (addition) (and & for intersection). So, if there were to be a dict union operator, the symbol should be | rather than +. ---------------------------------------------------------------------- Comment By: troy melhase (troy_melhase) Date: 2004-03-16 19:22 Message: Logged In: YES user_id=548370 1) adds new syntax but not new functionality 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() The same is true of list.extend and list.append. After years of writing python code, I still forget that I can't add dictionaries to one another, which for me at least, violates the principle of least surprise. The patch doesn't provide the same behavior as the update method; when adding two dicts, a third dict is returned. Only in the case of augmented assignment is the update method similar. Moreover, every other basic container-ish type supports addition of the same type (or similar type). It seems to me that the dict type is missing a feature, not the other way around. 2) concept doesn't usefully extend to - and * I could see it extending to -, but you're right that it doesn't extend to *. 5) possibly confusing given the | operator is used for sets I don't understand this point, could you elaborate? 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. But unlike list addition, dict addition is commutative: >>> d = {2:3} >>> e = {4:5} >>> d + e == e + d True Thanks for looking; could you advise again? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-16 11:08 Message: Logged In: YES user_id=80475 I recommend against this for several reasons: 1) adds new syntax but not new functionality 2) concept doesn't usefully extend to - and * 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() 4) no use cases showing the + operator to be more expressive 5) possibly confusing given the | operator is used for sets 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. If for some reason this gets approved, please assign back to me for implementation, documentation, and unittests. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 From naajjlcohujii at maine.rr.com Wed Mar 17 03:14:59 2004 From: naajjlcohujii at maine.rr.com ( Dorsey) Date: Wed Mar 17 03:19:08 2004 Subject: [Patches] Re: Your orzder In-Reply-To: References: Message-ID: <790617821.18734336@aamrwos> chairmen kingfisher eighteen namesake afloat muck crimea humidistat bestselling bear analgesic cater impressible quartzite d quicklime skimp shortage postage buzzard hospitable shawl colombia rumford rural inquisitor require splashy simply meditate elm cyril trailblaze tapir agnes ectopic bimini reparation hieronymus poughkeepsie court bitumen colon levitate quantify convolve yore poynting pr seaward botulism depreciable whistle priori rater dooley ineffable amazon naturopath apprise artwork district wet gunmen hecatomb checkpoint melville deltoid selma turnabout forgery metropolitan tamarind schuylkill interject boise elder bonito sidereal invitee felicitous chalcocite dickinson reveal parenthesis spring leaky naked -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040317/b031a263/attachment-0001.html From noreply at sourceforge.net Wed Mar 17 03:45:45 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 03:45:56 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by jiwon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 17:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Wed Mar 17 04:46:55 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 04:47:08 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-15 10:48 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Hye-Shik Chang (perky) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 18:46 Message: Logged In: YES user_id=55188 The code looks solid and fine for me. And it runs perfectly. BTW, it doesn't seem to accellate over-all speed much according to pystone and pybench on non-debug build while it absolutely accellates about 3~5% in debug build on my machine. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:25 Message: Logged In: YES user_id=80475 Hye-Shik, can you give this a second review? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-16 12:15 Message: Logged In: YES user_id=33168 Thanks Tim. I saw it late last night and didn't re-read today. Too much PSF work, not enough coding. :-( ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-16 12:12 Message: Logged In: YES user_id=31435 Neal, study the comment following that code. Sick code could have reduced the size of the dict as a side effect of calling PyObject_RichCompareBool(); that would make ma_mask smaller than it was at the start of the loop. Nothing about a dict can be assumed invariant across anything that may release the GIL or call back into Python (except for the address of the dict object). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-16 12:05 Message: Logged In: YES user_id=33168 Looking at the patch now, but I noticed a seemingly useless comparison in characterize around line 1328 (not related to your work): if (cmp > 0 || i > a->ma_mask || a->ma_table[i].me_value == NULL) I see no way for i > a->ma_mask based off the for loop (i <= a->ma_mask). Am I missing something? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-16 10:15 Message: Logged In: YES user_id=80475 Added a revised patch that can save even more work. If the previously freed dictionary has mp->mp_fill==0, then the memset() initialization can be skipped. Instrumenting this (against several different apps and the test suite) shows that 4 out of 5 mallocs are saved and the 1 in 10 memsets are saved. The counts fall off by about a quarter if the size of the freelist is cut in half (suggesting that 60 to 100 freedicts is plenty). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 17:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 13:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Wed Mar 17 06:00:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 06:00:31 2004 Subject: [Patches] [ python-Patches-917095 ] dict type concat function Message-ID: Patches item #917095, was opened at 2004-03-16 00:33 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: None Priority: 5 Submitted By: troy melhase (troy_melhase) Assigned to: Nobody/Anonymous (nobody) Summary: dict type concat function Initial Comment: Adds a function to dictobject.c that allows python expressions like: >>> d = {2:3} >>> d += {4:5} >>> d {2: 3, 4: 5} and like: >>> d = {2:3} >>> e = d + {6:7} >>> e {2: 3, 6: 7} A few points: * I don't know much C, and this patch is probably implemented much more appropriately by someone who does * I don't know if there's a good reason that the dict type doesn't already supply this; if that's the case, I'd be interested to know why not * Lib/test/test_builtin.py fails (as it should) after applying this patch Please advise, and thanks! ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 06:00 Message: Logged In: YES user_id=80475 The non-commutativity issue also applies to the | operator. I wouldn't spend time on a second patch unless there were significant new functionality, compelling use cases, a natural readability for expressions involving the operator, and an approach that suggests non-commutative update behavior. IOW, I wouldn't spend time on a second patch ;-) ---------------------------------------------------------------------- Comment By: troy melhase (troy_melhase) Date: 2004-03-17 02:33 Message: Logged In: YES user_id=548370 Doh. You're right about it being not commutative. I hadn't considered the example you provided. I agree that the | operator makes more sense. I'll close this patch until I'm up to implementing that. Thanks, Terry, Raymond! ---------------------------------------------------------------------- Comment By: Terry J. Reedy (tjreedy) Date: 2004-03-16 14:34 Message: Logged In: YES user_id=593130 I am usually big on orthogonality, but it never occurred to me that dicts should be addible, although sets are. What should be the result of {1:2} + {1:3}? If not something like {1:set (2,3)}, the operator is *not* commutative. The reference to sets and | is that | is used, I believe, for set union (addition) (and & for intersection). So, if there were to be a dict union operator, the symbol should be | rather than +. ---------------------------------------------------------------------- Comment By: troy melhase (troy_melhase) Date: 2004-03-16 14:22 Message: Logged In: YES user_id=548370 1) adds new syntax but not new functionality 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() The same is true of list.extend and list.append. After years of writing python code, I still forget that I can't add dictionaries to one another, which for me at least, violates the principle of least surprise. The patch doesn't provide the same behavior as the update method; when adding two dicts, a third dict is returned. Only in the case of augmented assignment is the update method similar. Moreover, every other basic container-ish type supports addition of the same type (or similar type). It seems to me that the dict type is missing a feature, not the other way around. 2) concept doesn't usefully extend to - and * I could see it extending to -, but you're right that it doesn't extend to *. 5) possibly confusing given the | operator is used for sets I don't understand this point, could you elaborate? 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. But unlike list addition, dict addition is commutative: >>> d = {2:3} >>> e = {4:5} >>> d + e == e + d True Thanks for looking; could you advise again? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-16 06:08 Message: Logged In: YES user_id=80475 I recommend against this for several reasons: 1) adds new syntax but not new functionality 2) concept doesn't usefully extend to - and * 3) writing d.update(x) is clearer and more flexible -- in Py2.4, update() can take the same arguments as dict() 4) no use cases showing the + operator to be more expressive 5) possibly confusing given the | operator is used for sets 6) {1:4}+{1:7} is not commutative and unlike list addition, there is no underlying order that makes the non- commutativity obvious. IOW, the operator may introduce a new class of hard to find bugs. If for some reason this gets approved, please assign back to me for implementation, documentation, and unittests. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=917095&group_id=5470 From noreply at sourceforge.net Wed Mar 17 10:59:39 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 12:26:33 2004 Subject: [Patches] [ python-Patches-918101 ] tarfile.py enhancements Message-ID: Patches item #918101, was opened at 2004-03-17 16:59 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918101&group_id=5470 Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py enhancements Initial Comment: I still develop tarfile.py sporadically on a separate branch (http://www.gustaebel.de/lars/tarfile/), and so there are two features from this branch that I'd like to propose for inclusion in Python's tarfile.py: 1. Overcoming the 8GB file size limit (8GB-limit.patch) At the moment it is not possible to add files to a tar archive that exceed 8GB size. Although this is POSIX compliant, GNU tar offers an extension header for largefiles that encodes file sizes in an 88-bit number instead of the common 11-digits octal number. Like all other GNU extensions in tarfile.py, this feature is turned on and off using the TarFile.posix attribute. 2. Automatic compression detection for the stream interface (stream-detect-compr.patch) tarfile.py's stream interface (which can be used to access tape devices or simply read a tar from stdin) is a bit difficult to use because it's not able to detect whether an archive is compressed or not. Compression has to be explicitly specified using mode ("r|", "r|gz", "r|bz2"). The patch introduces a fourth mode "r|*" that makes automatic detection possible. Both patches are not vitally important, but especially the 8GB-patch is useful IMO. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918101&group_id=5470 From noreply at sourceforge.net Wed Mar 17 12:40:17 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 12:40:23 2004 Subject: [Patches] [ python-Patches-896011 ] pdb enhancements and bug fixes Message-ID: Patches item #896011, was opened at 2004-02-12 12:07 Message generated for change (Settings changed) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=896011&group_id=5470 >Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Ilya Sandler (isandler) Assigned to: Nobody/Anonymous (nobody) Summary: pdb enhancements and bug fixes Initial Comment: Bug fixes B1) Execute script in its own namespace (script was executed in pdb.py name space. E.g one could do print line_prefix in your script and one would get pdb's line_prefix variable (and I guess if was possible to accidentally stomp on pdb internals from the script)) B2) Remove pdb.py's path from sys.path. (pdb.py path was not removed from sys.path.. While normally this is not a problem, it CAN cause hard to diagnose problems. e.g. if your script needs to override some of std modules and you put them in some location and set PYTHONPATH accordingly, then everything works without pdb, but when you run under pdb (which is normally located with the rest of python std libs) your overriding suddenly breaks) This addresses CVS Bug report #751124, part 2 B3) Fix handling of breakpoints "file_name:line_no" (pdb was failing to set the breakpoint specified as "b file_name:100" when file_name did not end with ".py", and was not an absolute path.This is quite bad as many scripts commonly do not have ".py" suffix... In at least some cases, (e.g when running under emacs gud) user can't workaround this problem, as the "break" command is issued by another program..) Enhancements: E1) Now pdb gets directly to the 1st line of script (the CVS version of pdb required doing "step" twice before you get to the first line of your code: both ugly and annoying) E2) Restart the program (and preserve debugger state, such as breakpoints) if program being debugged exits (CVS version was exitting on program's exitNow debugger does not die on script's exit, taking all the settings with it) E3) Enter post-mortem debugging if the program throws an uncaught exception (CVS version was simply dying in this case This addresses CVS bug report #751124, part 1) All changes apply to the command line interface of pdb.py Ie. only when pdb.py invoked as "pdb.py script" The only exception is B3 which fixes a bug in do_break() ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2004-03-05 11:28 Message: Logged In: YES user_id=971153 I am attaching a new version of the patch. Changes include: 1) the new patch is against the latest pdb.py v 1.67 2) it allows to debug code protected by if __name__== '__main__' conditional (the first version of the patch did not set __name__, so this code was not reachable).... Also, is there anything I can do to help with consideration of this patch? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=896011&group_id=5470 From noreply at sourceforge.net Wed Mar 17 13:40:09 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 13:40:23 2004 Subject: [Patches] [ python-Patches-918212 ] XHTML support for webchecker.py Message-ID: Patches item #918212, was opened at 2004-03-17 13:40 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918212&group_id=5470 Category: Demos and tools Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Chris Herborth (docoverlord) Assigned to: Nobody/Anonymous (nobody) Summary: XHTML support for webchecker.py Initial Comment: Here's a small patch that adds support for the 'id' attribute ('name' is deprecated in XHTML) to Tools/webchecker. README and webchecker.py are affected. Tested over approx. 26,000 lines of XHTML. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918212&group_id=5470 From noreply at sourceforge.net Wed Mar 17 17:19:38 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 17:19:49 2004 Subject: [Patches] [ python-Patches-916251 ] Create a freelist for dictionaries Message-ID: Patches item #916251, was opened at 2004-03-14 20:48 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed Resolution: None Priority: 5 Submitted By: Raymond Hettinger (rhettinger) Assigned to: Hye-Shik Chang (perky) Summary: Create a freelist for dictionaries Initial Comment: Neal, here is a simple implementation of freelists for dictionaries. I've not yet measured it across multiple applications and have no basis for knowing what a good maximum size should be (it's currently set at 100 which costs about 6K of memory when full). Right now, it only saves malloc time. It would great to also save some on the other initialization steps. The trick would be finding a way for delloc() to a little more work to save correspondingly more in PyDict_New(). Any review, ideas, independent timings, or creative thinking are welcome. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 17:19 Message: Logged In: YES user_id=80475 Thanks. Applying along with some other inner loop other optimizations. PyStone only measures a very narrow subset of python. PyBench is broader, but measures only a few specific actions. Also, in this case, I didn't expect a huge speedup -- dictionary creation is much less significant than the cost of using them. Also, dict creation doesn't occur nearly as often as tuple creation. Still, when it kicks in (which is most of the time), it almost always saves cycles. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 04:46 Message: Logged In: YES user_id=55188 The code looks solid and fine for me. And it runs perfectly. BTW, it doesn't seem to accellate over-all speed much according to pystone and pybench on non-debug build while it absolutely accellates about 3~5% in debug build on my machine. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 01:25 Message: Logged In: YES user_id=80475 Hye-Shik, can you give this a second review? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:15 Message: Logged In: YES user_id=33168 Thanks Tim. I saw it late last night and didn't re-read today. Too much PSF work, not enough coding. :-( ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-15 22:12 Message: Logged In: YES user_id=31435 Neal, study the comment following that code. Sick code could have reduced the size of the dict as a side effect of calling PyObject_RichCompareBool(); that would make ma_mask smaller than it was at the start of the loop. Nothing about a dict can be assumed invariant across anything that may release the GIL or call back into Python (except for the address of the dict object). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-15 22:05 Message: Logged In: YES user_id=33168 Looking at the patch now, but I noticed a seemingly useless comparison in characterize around line 1328 (not related to your work): if (cmp > 0 || i > a->ma_mask || a->ma_table[i].me_value == NULL) I see no way for i > a->ma_mask based off the for loop (i <= a->ma_mask). Am I missing something? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 20:15 Message: Logged In: YES user_id=80475 Added a revised patch that can save even more work. If the previously freed dictionary has mp->mp_fill==0, then the memset() initialization can be skipped. Instrumenting this (against several different apps and the test suite) shows that 4 out of 5 mallocs are saved and the 1 in 10 memsets are saved. The counts fall off by about a quarter if the size of the freelist is cut in half (suggesting that 60 to 100 freedicts is plenty). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-15 03:47 Message: Logged In: YES user_id=80475 >> There could be a problem in that the element's are freed. Sure they are. The dict isn't put on the freelist until *after* all the elements are freed by dealloc() and the ma_table is freed. All that is left is the shell that includes the ma_smalltable. >> would be better to free older dict refs No, no refs are kept. The shells are empty and undifferentiated, so age doesn't matter (just like the freelist scheme for tuples and frames). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2004-03-14 23:30 Message: Logged In: YES user_id=33168 There could be a problem in that the elements aren't freed. d = {} k, v = 1, 2 # create weakref to k and/or v d[k] = v del d The weakref callback wouldn't be called in this case, right? What are your thoughts on memory reclaimation in general? I don't know anything about the internals of dict, but if it's resized very large, should the entries be reduced? Doing more thinking out loud...I wonder if it would be better to free older dict refs when the table became full? I think the answer may also depend on size, memory locations (pages used), etc. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=916251&group_id=5470 From noreply at sourceforge.net Wed Mar 17 20:50:49 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 17 20:51:09 2004 Subject: [Patches] [ python-Patches-918462 ] simple "is" speedup Message-ID: Patches item #918462, was opened at 2004-03-17 19:50 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Raymond Hettinger (rhettinger) Summary: simple "is" speedup Initial Comment: All this "is" vs "==" discussion led me to look at ceval.c. The attached patch seems to speed up "is" and "is not" comparisons - saving a function call to do a simple pointer comparison for non-integer arguments. The test suite passes, but it's been quite awhile since I messed around with the interpreter code, so I thought I ought to have another pair of eyeballs check it out... ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 From 60hdneh at ya.com Thu Mar 18 01:28:25 2004 From: 60hdneh at ya.com (Herbert Hines) Date: Thu Mar 18 06:25:58 2004 Subject: [Patches] i Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040318/bd935237/attachment.html From bleendra at pakvalley.com Fri Mar 19 07:51:01 2004 From: bleendra at pakvalley.com (Helene Maddox) Date: Thu Mar 18 16:52:00 2004 Subject: [Patches] you could be john holmes Message-ID: Add another few inches to your meat and then REALLY lay the PIPE to her http://tradesman.shzze.com/vp5 Click here if you do not wish to be invited again: http://needtofindlovechat.com/remove/?oc=50791111 take off http://findingunity.com/remove/?oc=xxxxyyyy compositor comrade solecism phenomenon speak hoop fashion variant breech ego hippy 2 doleful eire d leakage studious tile platinum dragonfly abate bundestag comply babysitter baneful canterbury matchmake hyades 1 From noreply at sourceforge.net Thu Mar 18 23:00:42 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 18 23:00:46 2004 Subject: [Patches] [ python-Patches-919256 ] PEP-008: Update Naming Conventions Message-ID: Patches item #919256, was opened at 2004-03-18 23:00 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Kurt B. Kaiser (kbk) Assigned to: Nobody/Anonymous (nobody) Summary: PEP-008: Update Naming Conventions Initial Comment: Clarify and standardize the format for names of modules, functions, methods, and instance variables. Consistent, I hope, with discussion on python-dev http://mail.python.org/pipermail/python-dev/2004-March/043257.html http://mail.python.org/pipermail/python-dev/2004-March/043259.html ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 From noreply at sourceforge.net Thu Mar 18 23:27:52 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 18 23:28:14 2004 Subject: [Patches] [ python-Patches-914340 ] gzip.GzipFile to accept stream as fileobj. Message-ID: Patches item #914340, was opened at 2004-03-11 13:45 Message generated for change (Comment added) made by belyi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 Category: Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Igor Belyi (belyi) Assigned to: Nobody/Anonymous (nobody) Summary: gzip.GzipFile to accept stream as fileobj. Initial Comment: When gzip.GzipFile is initialized with a fileobj which does not have tell() and seek() methods (non-rewinding stream) it throws exception. The interesting thing is that it doesn't have to. The following patch updates gzip.py to allow any stream with just a read() method to be used. This is helpful if you want to be able to do something like: gzip.GzipFile(fileobj=urllib.urlopen("file:///README.gz")).readlines() or use GzipFile with sys.stdin stream. But keep in mind that seek() and rewind() methond of the GzipFile() won't for such stream even with the patch. Igor ---------------------------------------------------------------------- >Comment By: Igor Belyi (belyi) Date: 2004-03-18 23:27 Message: Logged In: YES user_id=995711 I thought I need to add a little bit more verbose explanation for the changes... Current implementation of GzipFile() uses tell() and seek() to scroll stream of data in the following 2 cases: 1. When EOF is reached and the last 8 bytes of the file contain checksum and uncompress data size 2. When after decompression there's left some 'unused_data' meaning that a stream may contains more than one compressed item. What my change does it introduces 2 helper buffers: 'inputbuf' which keeps read but unused data from the stream and 'last8' which keeps last 8 'used' bytes Plus, my change introduces helper method _read_internal() which is used instead of the direct call to self.fileobj.read(). In this method data from the stream are read as needed with the call to self.fileobj.read() and correct values of 'inputbuf' and ''last8' are maintained. When case 1 above happen we use 'last8' buffer to read checksum and size. When case 2 above happen we add value of the 'unused_data' to inputbuf. There's one more instance of the self.fileobj.seek() call left in rewind() method but it is used only when rewind() or seek() methods of GzipFile class are used. And it won't be logical to expect those methods to work if the underlying fileobj does not support them. Igor ---------------------------------------------------------------------- Comment By: Igor Belyi (belyi) Date: 2004-03-11 15:04 Message: Logged In: YES user_id=995711 Previous revision of the patch does not work correctly with mutliple compressed members in one stream. I've updated the patch file. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 From noreply at sourceforge.net Thu Mar 18 23:54:14 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 18 23:54:21 2004 Subject: [Patches] [ python-Patches-919256 ] PEP-008: Update Naming Conventions Message-ID: Patches item #919256, was opened at 2004-03-18 23:00 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Kurt B. Kaiser (kbk) >Assigned to: Kurt B. Kaiser (kbk) Summary: PEP-008: Update Naming Conventions Initial Comment: Clarify and standardize the format for names of modules, functions, methods, and instance variables. Consistent, I hope, with discussion on python-dev http://mail.python.org/pipermail/python-dev/2004-March/043257.html http://mail.python.org/pipermail/python-dev/2004-March/043259.html ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2004-03-18 23:54 Message: Logged In: YES user_id=6380 Thanks! Looks good, except that the alternate style for functions and methods is "mixedCase", not "CamelCase". (We may have our definition of CamelCase mixed up, but the point is that the initial letter is lower case.) Go ahead and check it in (if people disagree they can check in an improvement). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 From noreply at sourceforge.net Fri Mar 19 01:21:48 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 19 01:21:56 2004 Subject: [Patches] [ python-Patches-919299 ] API Ref: PyErr_SetInterrupt not Obsolete Message-ID: Patches item #919299, was opened at 2004-03-19 01:21 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919299&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Kurt B. Kaiser (kbk) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: API Ref: PyErr_SetInterrupt not Obsolete Initial Comment: The API document Section 4. claims that this function is obsolete, but it is used in thread.interrupt_main(), which in turn is used in IDLE. Attached patch removes the 'obsolete' statement. If there is some reason that it is going to disappear, then we have to figure something out for thread.interrupt_main. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919299&group_id=5470 From noreply at sourceforge.net Fri Mar 19 08:37:57 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 19 08:38:08 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by quiver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-03-19 22:37 Message: Logged In: YES user_id=671362 Hi, Jiwon. This is another bug candidate. If you use genexp with iterators, it doesn't work well. # test Perky's previous post using iterators >>> list((x,y) for x in iter('abcd') for y in iter('abcd')) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Thanks. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 17:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Fri Mar 19 09:14:40 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 19 09:15:11 2004 Subject: [Patches] [ python-Patches-914340 ] gzip.GzipFile to accept stream as fileobj. Message-ID: Patches item #914340, was opened at 2004-03-11 13:45 Message generated for change (Comment added) made by belyi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 Category: Modules Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Igor Belyi (belyi) Assigned to: Nobody/Anonymous (nobody) Summary: gzip.GzipFile to accept stream as fileobj. Initial Comment: When gzip.GzipFile is initialized with a fileobj which does not have tell() and seek() methods (non-rewinding stream) it throws exception. The interesting thing is that it doesn't have to. The following patch updates gzip.py to allow any stream with just a read() method to be used. This is helpful if you want to be able to do something like: gzip.GzipFile(fileobj=urllib.urlopen("file:///README.gz")).readlines() or use GzipFile with sys.stdin stream. But keep in mind that seek() and rewind() methond of the GzipFile() won't for such stream even with the patch. Igor ---------------------------------------------------------------------- >Comment By: Igor Belyi (belyi) Date: 2004-03-19 09:14 Message: Logged In: YES user_id=995711 I thought I need to add a little bit more verbose explanation for the changes... Current implementation of GzipFile() uses tell() and seek() to scroll stream of data in the following 2 cases: 1. When EOF is reached and the last 8 bytes of the file contain checksum and uncompress data size 2. When after decompression there's left some 'unused_data' meaning that a stream may contains more than one compressed item. What my change does it introduces 2 helper buffers: 'inputbuf' which keeps read but unused data from the stream and 'last8' which keeps last 8 'used' bytes Plus, my change introduces helper method _read_internal() which is used instead of the direct call to self.fileobj.read(). In this method data from the stream are read as needed with the call to self.fileobj.read() and correct values of 'inputbuf' and ''last8' are maintained. When case 1 above happen we use 'last8' buffer to read checksum and size. When case 2 above happen we add value of the 'unused_data' to inputbuf. There's one more instance of the self.fileobj.seek() call left in rewind() method but it is used only when rewind() or seek() methods of GzipFile class are used. And it won't be logical to expect those methods to work if the underlying fileobj does not support them. Igor ---------------------------------------------------------------------- Comment By: Igor Belyi (belyi) Date: 2004-03-18 23:27 Message: Logged In: YES user_id=995711 I thought I need to add a little bit more verbose explanation for the changes... Current implementation of GzipFile() uses tell() and seek() to scroll stream of data in the following 2 cases: 1. When EOF is reached and the last 8 bytes of the file contain checksum and uncompress data size 2. When after decompression there's left some 'unused_data' meaning that a stream may contains more than one compressed item. What my change does it introduces 2 helper buffers: 'inputbuf' which keeps read but unused data from the stream and 'last8' which keeps last 8 'used' bytes Plus, my change introduces helper method _read_internal() which is used instead of the direct call to self.fileobj.read(). In this method data from the stream are read as needed with the call to self.fileobj.read() and correct values of 'inputbuf' and ''last8' are maintained. When case 1 above happen we use 'last8' buffer to read checksum and size. When case 2 above happen we add value of the 'unused_data' to inputbuf. There's one more instance of the self.fileobj.seek() call left in rewind() method but it is used only when rewind() or seek() methods of GzipFile class are used. And it won't be logical to expect those methods to work if the underlying fileobj does not support them. Igor ---------------------------------------------------------------------- Comment By: Igor Belyi (belyi) Date: 2004-03-11 15:04 Message: Logged In: YES user_id=995711 Previous revision of the patch does not work correctly with mutliple compressed members in one stream. I've updated the patch file. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=914340&group_id=5470 From maureenaltavilla at realwebsurf.com Fri Mar 19 14:53:55 2004 From: maureenaltavilla at realwebsurf.com (Consuelo Frey) Date: Fri Mar 19 11:49:51 2004 Subject: [Patches] john holmes ..... YOU Message-ID: Add another few inches to your meat and then REALLY lay the PIPE to her http://captain.shzze.com/vp5 Click here if you do not wish to be invited again: http://needtofindlovechat.com/remove/?oc=50791111 take off http://findingunity.com/remove/?oc=xxxxyyyy cation canopy doubleday quagmire about annular quiz gaberones optoelectronic edgy deluxe wooster 2 caliper basilar fredericks gyroscope arrow cypress bellini congestion olivia shrill falter trepidation synchronism tenfold contributor harrisburg external composure gladiator pliny orgasm briefcase uk codeposit 7 From noreply at sourceforge.net Fri Mar 19 12:51:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 19 12:51:13 2004 Subject: [Patches] [ python-Patches-919256 ] PEP-008: Update Naming Conventions Message-ID: Patches item #919256, was opened at 2004-03-18 23:00 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Kurt B. Kaiser (kbk) Assigned to: Kurt B. Kaiser (kbk) Summary: PEP-008: Update Naming Conventions Initial Comment: Clarify and standardize the format for names of modules, functions, methods, and instance variables. Consistent, I hope, with discussion on python-dev http://mail.python.org/pipermail/python-dev/2004-March/043257.html http://mail.python.org/pipermail/python-dev/2004-March/043259.html ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-19 12:51 Message: Logged In: YES user_id=764593 Under modules, should there be a parenthetical comment saying either (1) If a module name does include capitals, then it must be named with capitals during import? (2) Many existing module names follow an older CapWords convention, particularly if they export a single class or wrap a C module. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-03-18 23:54 Message: Logged In: YES user_id=6380 Thanks! Looks good, except that the alternate style for functions and methods is "mixedCase", not "CamelCase". (We may have our definition of CamelCase mixed up, but the point is that the initial letter is lower case.) Go ahead and check it in (if people disagree they can check in an improvement). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 From foreniela at footyforecast.every1.net Sat Mar 20 02:46:26 2004 From: foreniela at footyforecast.every1.net (Jill Clark) Date: Fri Mar 19 23:42:00 2004 Subject: [Patches] Refinance-now even with bad-credit Message-ID: Got bad-credit? Let us help you refinance your home Lowest In.terest Rates Available! Get a Pre-Approval in 48 hours for F.RE.E Refinancing for the Self-Employed SAVE $100-$400 per month and skip ONE payment Our easy application only takes 1 minute. http://refinancemade-easy.com/?partid=wh6 The above gift or special offer was sent to you as a subscriber of Direct Media. We will continue to bring you valuable offers on products and services that interest you most. To modify your future preference with us: http://refinancemade-easy.com/st.html From noreply at sourceforge.net Sat Mar 20 01:53:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 01:53:14 2004 Subject: [Patches] [ python-Patches-919256 ] PEP-008: Update Naming Conventions Message-ID: Patches item #919256, was opened at 2004-03-18 23:00 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Kurt B. Kaiser (kbk) Assigned to: Kurt B. Kaiser (kbk) Summary: PEP-008: Update Naming Conventions Initial Comment: Clarify and standardize the format for names of modules, functions, methods, and instance variables. Consistent, I hope, with discussion on python-dev http://mail.python.org/pipermail/python-dev/2004-March/043257.html http://mail.python.org/pipermail/python-dev/2004-March/043259.html ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2004-03-20 01:53 Message: Logged In: YES user_id=149084 pep-0008.txt 1.21 ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-19 12:51 Message: Logged In: YES user_id=764593 Under modules, should there be a parenthetical comment saying either (1) If a module name does include capitals, then it must be named with capitals during import? (2) Many existing module names follow an older CapWords convention, particularly if they export a single class or wrap a C module. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-03-18 23:54 Message: Logged In: YES user_id=6380 Thanks! Looks good, except that the alternate style for functions and methods is "mixedCase", not "CamelCase". (We may have our definition of CamelCase mixed up, but the point is that the initial letter is lower case.) Go ahead and check it in (if people disagree they can check in an improvement). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919256&group_id=5470 From 72mujytg at ya.com Sat Mar 20 10:15:24 2004 From: 72mujytg at ya.com (Louisa Jeffers) Date: Sat Mar 20 07:13:01 2004 Subject: [Patches] agave Message-ID: <3$$p27p$5ovzvb3-$ux-z$p0q8@t9qy.7nw06rp> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040320/2059f875/attachment.html From noreply at sourceforge.net Sat Mar 20 07:18:32 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 07:18:35 2004 Subject: [Patches] [ python-Patches-920037 ] email/__init__.py: Parse headersonly Message-ID: Patches item #920037, was opened at 2004-03-20 12:18 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920037&group_id=5470 Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Chris Mayo (cjmayo) Assigned to: Nobody/Anonymous (nobody) Summary: email/__init__.py: Parse headersonly Initial Comment: The attached simple patch brings the useful headersonly parameter up to the message_from_string function to optionally parse the headers of messages without bodies. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920037&group_id=5470 From noreply at sourceforge.net Sat Mar 20 12:37:05 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 12:37:12 2004 Subject: [Patches] [ python-Patches-795531 ] license inconsistency Message-ID: Patches item #795531, was opened at 2003-08-26 14:08 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=795531&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Brian Gough (briangough) Assigned to: Nobody/Anonymous (nobody) Summary: license inconsistency Initial Comment: there is a discrepancy between the format of the dates in copyright.tex and in license.tex (2001,2002,2003 vs 2001-2003), which fails on a strict interpretation of the license ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 12:37 Message: Logged In: YES user_id=31435 Thanks. I changed copyright.tex instead -- changing the license is a major event for lots of people. Doc/commontex/copyright.tex; new revision: 1.2 Doc/commontex/license.tex; new revision: 1.4 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=795531&group_id=5470 From noreply at sourceforge.net Sat Mar 20 12:45:05 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 12:45:14 2004 Subject: [Patches] [ python-Patches-918462 ] simple Message-ID: Patches item #918462, was opened at 2004-03-17 20:50 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Raymond Hettinger (rhettinger) >Summary: simple Initial Comment: All this "is" vs "==" discussion led me to look at ceval.c. The attached patch seems to speed up "is" and "is not" comparisons - saving a function call to do a simple pointer comparison for non-integer arguments. The test suite passes, but it's been quite awhile since I messed around with the interpreter code, so I thought I ought to have another pair of eyeballs check it out... ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 12:45 Message: Logged In: YES user_id=31435 Well, there's little question that this will speed "is" and "is not", but it also slows all other cases by the cost of the switch-and-branch to determine that they're not the favored cases. So why should we believe that speeding "is" and "is not" is more important than slowing other cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 From noreply at sourceforge.net Sat Mar 20 13:27:08 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 13:27:16 2004 Subject: [Patches] [ python-Patches-918462 ] simple Message-ID: Patches item #918462, was opened at 2004-03-17 20:50 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Raymond Hettinger (rhettinger) Summary: simple Initial Comment: All this "is" vs "==" discussion led me to look at ceval.c. The attached patch seems to speed up "is" and "is not" comparisons - saving a function call to do a simple pointer comparison for non-integer arguments. The test suite passes, but it's been quite awhile since I messed around with the interpreter code, so I thought I ought to have another pair of eyeballs check it out... ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-20 13:27 Message: Logged In: YES user_id=80475 Even "is" and "is not" are not helped by more than a couple of cycles. This fragment essentially inlines part of code for cmp_outcome(). Only the function call is saved. It does slow down other code paths by introducing an unpredictable branch. If the inlining were considered important, then the whole of cmp_outcome() should be inlined. Then, all comparisons save a single call/return pair. The cost is further increasing the size of the eval loop. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-20 12:45 Message: Logged In: YES user_id=31435 Well, there's little question that this will speed "is" and "is not", but it also slows all other cases by the cost of the switch-and-branch to determine that they're not the favored cases. So why should we believe that speeding "is" and "is not" is more important than slowing other cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 From noreply at sourceforge.net Sat Mar 20 15:24:22 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 15:24:28 2004 Subject: [Patches] [ python-Patches-895445 ] hmac.HMAC.copy() speedup Message-ID: Patches item #895445, was opened at 2004-02-12 00:28 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=895445&group_id=5470 Category: Library (Lib) Group: Python 2.3 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: hmac.HMAC.copy() speedup Initial Comment: Hi, I'm using python 2.3.2, on WinXP. This is my first submitted patch, let me know if I'm doing anything wrong.. The current hmac.HMAC.copy() constructs a new instance first, then overwrites it. The hmac._strxor() function called in the constructor is slow. HMAC is used in protocols like TLS & IPSEC where you want to MAC each packet or record sent, so you'd want to construct an HMAC object once, then copy() it for each record. I'm writing a TLS implementation, and the patch here gives me ~30% increase in throughput. Trevor ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 15:24 Message: Logged In: YES user_id=31435 It's a decent idea -- thanks! The mechanics of the patch were fine. It's not good to use None as a secret backdoor argument, since passing a computed None by mistake is a common program flaw, and getting back an insane HMAC object then is a poor outcome. So the patch I checked in uses a different gimmick (a unique module-private object -- such a thing cannot be passed by mistake). Lib/hmac.py; new revision: 1.8 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=895445&group_id=5470 From noreply at sourceforge.net Sat Mar 20 15:41:01 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 15:41:04 2004 Subject: [Patches] [ python-Patches-920211 ] Patch submission for #876533 (potential leak in ensure_froml Message-ID: Patches item #920211, was opened at 2004-03-20 16:41 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920211&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Brian Leair (bleair2) Assigned to: Nobody/Anonymous (nobody) Summary: Patch submission for #876533 (potential leak in ensure_froml Initial Comment: Summary: (?) potential leak in ensure_fromlist (import.c) In the recursive call inside of ensure_fromlist () an early exit is taken if the recursive call returns 0. In this early exit case the object "all" needs to be Py_DECREF (). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920211&group_id=5470 From noreply at sourceforge.net Sat Mar 20 15:52:23 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 15:52:29 2004 Subject: [Patches] [ python-Patches-895445 ] hmac.HMAC.copy() speedup Message-ID: Patches item #895445, was opened at 2004-02-12 00:28 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=895445&group_id=5470 Category: Library (Lib) Group: Python 2.3 Status: Closed Resolution: Accepted Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: hmac.HMAC.copy() speedup Initial Comment: Hi, I'm using python 2.3.2, on WinXP. This is my first submitted patch, let me know if I'm doing anything wrong.. The current hmac.HMAC.copy() constructs a new instance first, then overwrites it. The hmac._strxor() function called in the constructor is slow. HMAC is used in protocols like TLS & IPSEC where you want to MAC each packet or record sent, so you'd want to construct an HMAC object once, then copy() it for each record. I'm writing a TLS implementation, and the patch here gives me ~30% increase in throughput. Trevor ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 15:52 Message: Logged In: YES user_id=31435 It's a decent idea -- thanks! The mechanics of the patch were fine. It's not good to use None as a secret backdoor argument, since passing a computed None by mistake is a common program flaw, and getting back an insane HMAC object then is a poor outcome. So the patch I checked in uses a different gimmick (a unique module-private object -- such a thing cannot be passed by mistake). Lib/hmac.py; new revision: 1.8 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-20 15:24 Message: Logged In: YES user_id=31435 It's a decent idea -- thanks! The mechanics of the patch were fine. It's not good to use None as a secret backdoor argument, since passing a computed None by mistake is a common program flaw, and getting back an insane HMAC object then is a poor outcome. So the patch I checked in uses a different gimmick (a unique module-private object -- such a thing cannot be passed by mistake). Lib/hmac.py; new revision: 1.8 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=895445&group_id=5470 From noreply at sourceforge.net Sat Mar 20 15:59:37 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 15:59:45 2004 Subject: [Patches] [ python-Patches-767600 ] Add a 'isotime' format to standard logging Message-ID: Patches item #767600, was opened at 2003-07-08 04:04 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=767600&group_id=5470 Category: Library (Lib) Group: None >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Vinay Sajip (vsajip) Summary: Add a 'isotime' format to standard logging Initial Comment: This patch adds a standard format string 'isotime' to go alongside 'asctime' in the standard logging module. The 'asctime' format is somewhat idiosyncratic (for instance the seconds,milliseconds part). 'isotime' produces UTC timestamps in ISO8601 format, that is %Y-%m-%dT%H:%M:%S.msecsZ e.g. 2003-07-08T18:02:34.012Z A good primer on ISO 8601 can be found at http://www.cl.cam.ac.uk/~mgk25/iso-time.html I'm not entirely sold on the name 'isotime' - an alternative would be 'stdtime'. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 15:59 Message: Logged In: YES user_id=31435 There's still no patch here, and it's heading on a year since this was opened, so closing as Invalid. If someone wants to "do something" here, please open a new report with an actual patch. ---------------------------------------------------------------------- Comment By: Vinay Sajip (vsajip) Date: 2003-07-13 19:25 Message: Logged In: YES user_id=308438 The current implementation is supposed to produce ISO8601 format by default, unless a specific date format is specified for the formatter. According to the above web page, the only way in which the current default format is non-standard is that no 'Z' is provided to indicate a UTC time; so I think the patch should not introduce a new 'stdtime' or 'isotime' format, but rather change the current formatting implementation so that the standard format is actually produced. Currently, the millisecond separator is a comma (like log4j); Markus Kuhn's above page indicates that either a comma or a period are acceptable for this. Of course, it may be that better support for standard date/time formats should not be implemented in logging, but in a general-purpose package which can be used not only by logging, but by other applications which need this functionality. Anthony, nice to see that you are taking another look at logging :-) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-07-08 04:35 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=767600&group_id=5470 From noreply at sourceforge.net Sat Mar 20 16:18:29 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 16:18:34 2004 Subject: [Patches] [ python-Patches-693221 ] fix bug 625698, speed up some comparisons Message-ID: Patches item #693221, was opened at 2003-02-25 16:57 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=693221&group_id=5470 Category: Core (C code) Group: Python 2.3 >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Michael Stone (mbrierst) Assigned to: Nobody/Anonymous (nobody) Summary: fix bug 625698, speed up some comparisons Initial Comment: This patch, patchrecursion GREATLY speeds up comparisons of recursive objects, fixing bug 625698. It works by throwing a new exception, a RecursionError, to break out of the current deeply nested comparison, and then redoing the comparison by the already established technique for recursive objects. I believe the only costs to normal non-recursive comparisons is two additional C comparisons. Backwards compatibility problems will occur only in code that attempts to handle ALL exceptions while doing comparisons. Of course, this is always a bad practice, so hopefully not much code does this. Also, most code probably doesn't attempt to handle exceptions within the actual comparison routines, instead catching them elsewhere, which will be fine. Another patch is also attached, patchequal, to handle some of the common cases where this comes up even faster. While it is true that python wants to have an == operator that does not define an equality relation, I believe that when == is not an equality relation python has few obligations to be nice to it. In particular, suppose we have an object NaN != NaN. I would say that (NaN,) == (NaN,) [NaN] == [NaN] {NaN:1} == {NaN:1} all intuitively still seem true. Isn't it obvious that if two lists contain the SAME objects, they are equal? That is, even though NaN does not equal itself, lists, tuples, and dicts have no obligation to respect the underlying == operator. I don't see any documentation saying they will respect it (though maybe I'm just missing it). patchequal assumes, ONLY for purposes of list tuple dict comparisons and dict lookups, that if id(a) == id(b), then a == b. Note there is some precedence for this patch, d = {NaN:1} NaN in d => True cmp(NaN, NaN) is 0 (though I suspect this is a mistake). This will cause backwards compatibility problems only for people using non-equality relation =='s, and probably not even for them, as they already probably use a trick to emulate this patch when they compare lists and such containing these weird objects. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 16:18 Message: Logged In: YES user_id=31435 Python 2.4 was changed so that recursive comparisons (all recursive comparisons) raise RuntimeError: maximum recursion depth exceeded in cmp so rejecting the patch (Python doesn't *want* to guess the intended meaning of a recursive compare, regardless of the speed at which it could make such a guess). Thanks for playing, though! Just find something useful next time. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=693221&group_id=5470 From noreply at sourceforge.net Sat Mar 20 17:03:34 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 17:03:40 2004 Subject: [Patches] [ python-Patches-508730 ] CGIHTTPServer execfile should save cwd Message-ID: Patches item #508730, was opened at 2002-01-25 18:41 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=508730&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Dan Grassi (dgrassi) Assigned to: Nobody/Anonymous (nobody) Summary: CGIHTTPServer execfile should save cwd Initial Comment: In CGIHTTPServer when running a script via execfile the cwd is not saved and restored. If the executed script changes the cwd subsequent requests will fail because self.translate_path in SimpleHTTPServer relies on the cwd. One fix is to suround the execfile call with save_cwd = os.getcwd() and os.chdir(save_cwd) Here is the proposed fix starting at line 254: + save_cwd = os.getcwd() try: try: sys.argv = [scriptfile] if '=' not in decoded_query: sys.argv.append(decoded_query) sys.stdout = self.wfile sys.stdin = self.rfile execfile(scriptfile, {"__name__": "__main__"}) finally: + os.chdir(save_cwd) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 17:03 Message: Logged In: YES user_id=31435 I checked this in, but have no idea how to test it. In part that's because the execfile path doesn't appear to be taken on Unixish *or* Windows systems. Is this entire block of code actually unreachable (== is there an OS now that doesn't have fork or popen2 or popen3?). Lib/CGIHTTPServer.py; new revision: 1.34 ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2003-10-24 08:14 Message: Logged In: YES user_id=11375 It looks like a bug worth fixing, and the proposed patch seems reasonable. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-05-16 20:47 Message: Logged In: YES user_id=357491 Since a patch is in the comments I am making this a patch instead of a bug. As for the idea, it seems good to me. Anyone else care to comment? ---------------------------------------------------------------------- Comment By: Dan Grassi (dgrassi) Date: 2002-01-25 18:55 Message: Logged In: YES user_id=366473 Here is the (hopefully) properly indented fix: + save_cwd = os.getcwd() try: try: sys.argv = [scriptfile] if '=' not in decoded_query: sys.argv.append(decoded_query) sys.stdout = self.wfile sys.stdin = self.rfile execfile(scriptfile, {"__name__": "__main__"}) finally: + os.chdir(save_cwd) ---------------------------------------------------------------------- Comment By: Dan Grassi (dgrassi) Date: 2002-01-25 18:48 Message: Logged In: YES user_id=366473 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=508730&group_id=5470 From noreply at sourceforge.net Sat Mar 20 17:29:38 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 17:30:00 2004 Subject: [Patches] [ python-Patches-508665 ] Improvement of cgi.parse_qsl function Message-ID: Patches item #508665, was opened at 2002-01-25 15:23 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=508665&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Christoph Zwerschke (cito) >Assigned to: Brett Cannon (bcannon) Summary: Improvement of cgi.parse_qsl function Initial Comment: I found the parsing function "parse_qsl" in the module "cgi" to have some flaws. Especially, empty names are allowed, even if empty values are explicitly disallowed. If the latter are allowed, "?name=" is accepted, while "?name" is ignored. Often you want to use links like "?logout" or "?help". This is not possible, even if empty values are explicitly allowed. Also, "strict parsing" objects to "?name=", while it ignores "?name=a=b=c". My improvement suggestion: ------------- use ---------- for name_value in pairs: if strict_parsing: nv = name_value.split('=', 2) if len(nv) != 2 or not len(nv[0]): raise ValueError, "bad query field: %s" % `name_value` else: nv = name_value.split('=', 1).append('') if not len(nv[0]): continue if len(nv[1]) or keep_blank_values: name = urllib.unquote(nv[0].replace('+', ' ')) value = urllib.unquote(nv[1].replace('+', ' ')) r.append((name, value)) ----------- instead of -------- for name_value in pairs: nv = name_value.split('=', 1) if len(nv) != 2: if strict_parsing: raise ValueError, "bad query field: %s" % `name_value` continue if len(nv[1]) or keep_blank_values: name = urllib.unquote(nv[0].replace('+', ' ')) value = urllib.unquote(nv[1].replace('+', ' ')) r.append((name, value)) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-20 17:29 Message: Logged In: YES user_id=31435 Brett, since you seemed to know something about this, how about closing it? You just got the honor of having our oldest open patch assigned to you . ---------------------------------------------------------------------- Comment By: Christoph Zwerschke (cito) Date: 2003-05-19 10:05 Message: Logged In: YES user_id=193957 The problem with empty names is still the same. Is this what you need? cvs diff cgi.py (in directory C:\Temp\python\python\dist\src\Lib\) Index: cgi.py ================================================ =================== RCS file: /cvsroot/python/python/dist/src/Lib/cgi.py,v retrieving revision 1.76 diff -r1.76 cgi.py 212,214c212,214 < nv = name_value.split('=', 1) < if len(nv) != 2: < if strict_parsing: --- > if strict_parsing: > nv = name_value.split('=', 2) > if len(nv) != 2 or not len(nv[0]): 216c216,221 < continue --- > else: > nv = name_value.split('=', 1) > if not len(nv[0]): > continue > if len(nv) != 2: > nv.append('') ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2003-05-16 20:41 Message: Logged In: YES user_id=357491 The issue of "name=" compared to "name=a=b=c" has changed; both are allowed under strict parsing while "name" is not. The isue with "name" not being made a key with a blank value is still there. Christoph, any chance you can create a patch against the CVS version of cgi? ---------------------------------------------------------------------- Comment By: Christoph Zwerschke (cito) Date: 2002-01-25 15:41 Message: Logged In: YES user_id=193957 -------- better use: ---------- 

for name_value in pairs:
    if strict_parsing:
        nv = name_value.split('=', 2)
        if len(nv) != 2 or not len(nv[0]):
            raise ValueError, "bad query field: %s" % 
`name_value`
    else:
        nv = name_value.split('=', 1)
        if not len(nv[0]):
            continue
        if len(nv) != 2:
            nv.append('')
    if len(nv[1]) or keep_blank_values:
        name = urllib.unquote(nv[0].replace('+', ' '))
        value = urllib.unquote(nv[1].replace('+', ' '))
        r.append((name, value))
---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=508665&group_id=5470 From noreply at sourceforge.net Sat Mar 20 18:02:41 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 18:02:46 2004 Subject: [Patches] [ python-Patches-790710 ] breakpoint command lists in pdb Message-ID: Patches item #790710, was opened at 2003-08-18 13:52 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=790710&group_id=5470 Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Gr?goire Dooms (dooms) Assigned to: Guido van Rossum (gvanrossum) Summary: breakpoint command lists in pdb Initial Comment: This patch enables the user to define a list of commands associated to a breakpoint. Those commands are executed whenever the given breakpoint makes the program stop execution. The "commands [bpnumber]" command works like in gdb. This patch is against the CVS HEAD, it has been tested with python 2.3. The documentation is updated too. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2004-03-20 18:02 Message: Logged In: YES user_id=6380 I'm all for this, but don't have the bandwidth to review it. Who does? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=790710&group_id=5470 From noreply at sourceforge.net Sat Mar 20 18:06:58 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 20 18:07:01 2004 Subject: [Patches] [ python-Patches-796772 ] CGIHTTPServer fix Message-ID: Patches item #796772, was opened at 2003-08-28 12:34 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=796772&group_id=5470 Category: Library (Lib) Group: Python 2.3 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: CGIHTTPServer fix Initial Comment: Here's a fix for the CGI server. The problem only occurs on systems that support os.fork(). When you invoke a CGI script with a query string and then subsequent invoke it without a query string (or with an empty query string), the second invocation gets passed the query string of the first invocation. This is because of the way os.environ is updated in the parent, but when there is no query string, the old query string doesn't get deleted. The solution is to update the os.environ in the child. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2004-03-20 18:06 Message: Logged In: YES user_id=6380 I believe the fix I checked in for SF 777848 solves this in a different way. Can either of the reviewers see any problem with that solution? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-09-10 02:12 Message: Logged In: YES user_id=21627 I agree with rhettinger. The original patch assumes that the platform has putenv/setenv, which it might not (if it doesn't, then the os.environ.update is not reflected in the C environment, and thus has no effect on os.execv). Whether or not os.environ needs to be copied is debatable; the copying has its cost, but avoiding the putenv calls would save some of the costs. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-08-31 10:48 Message: Logged In: YES user_id=80475 The first part of the patch makes sense to me. The second part updates os.environ which is shared between successive calls. I would have expected something like: childenv = os.environ.copy() childenv.update(env) . . . os.execve(scriptfile, args, childenv) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=796772&group_id=5470 From noreply at sourceforge.net Sun Mar 21 09:33:31 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 09:33:58 2004 Subject: [Patches] [ python-Patches-918462 ] simple Message-ID: Patches item #918462, was opened at 2004-03-17 19:50 Message generated for change (Comment added) made by montanaro You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Closed Resolution: Rejected Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Raymond Hettinger (rhettinger) Summary: simple Initial Comment: All this "is" vs "==" discussion led me to look at ceval.c. The attached patch seems to speed up "is" and "is not" comparisons - saving a function call to do a simple pointer comparison for non-integer arguments. The test suite passes, but it's been quite awhile since I messed around with the interpreter code, so I thought I ought to have another pair of eyeballs check it out... ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2004-03-21 08:33 Message: Logged In: YES user_id=44345 I spent a fair amount of time yesterday refining and running a shell script (attached) to compare the before and after times for various comparisons of simple objects. Here's the output: s = 'abc' operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.375 0.329 0.046 -12.3 s == 'abc' 0.491 0.493 -0.002 0.4 s > 'abc' 0.491 0.493 -0.002 0.4 s is 4 0.375 0.333 0.042 -11.2 s == 4 1.200 1.190 0.010 -0.8 s > 4 1.200 1.190 0.010 -0.8 s is -1001 0.378 0.332 0.046 -12.2 s == -1001 1.200 1.190 0.010 -0.8 s > -1001 1.200 1.180 0.020 -1.7 s is 34.7 0.370 0.325 0.045 -12.2 s == 34.7 1.620 1.590 0.030 -1.9 s > 34.7 1.600 1.590 0.010 -0.6 s is 'a b c' 0.369 0.328 0.041 -11.1 s == 'a b c' 0.475 0.476 -0.001 0.2 s > 'a b c' 0.559 0.563 -0.004 0.7 s is True 0.531 0.491 0.040 -7.5 s == True 1.400 1.390 0.010 -0.7 s > True 1.400 1.380 0.020 -1.4 s = 4 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.369 0.325 0.044 -11.9 s == 'abc' 1.200 1.190 0.010 -0.8 s > 'abc' 1.200 1.190 0.010 -0.8 s is 4 0.353 0.353 0.000 0.0 s == 4 0.352 0.355 -0.003 0.9 s > 4 0.354 0.350 0.004 -1.1 s is -1001 0.347 0.350 -0.003 0.9 s == -1001 0.350 0.353 -0.003 0.9 s > -1001 0.346 0.345 0.001 -0.3 s is 34.7 0.367 0.327 0.040 -10.9 s == 34.7 0.773 0.769 0.004 -0.5 s > 34.7 0.771 0.772 -0.001 0.1 s is 'a b c' 0.370 0.327 0.043 -11.6 s == 'a b c' 1.200 1.190 0.010 -0.8 s > 'a b c' 1.200 1.190 0.010 -0.8 s is True 0.534 0.492 0.042 -7.9 s == True 0.905 0.911 -0.006 0.7 s > True 0.904 0.913 -0.009 1.0 s = None operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.368 0.327 0.041 -11.1 s == 'abc' 0.962 0.950 0.012 -1.2 s > 'abc' 0.959 0.955 0.004 -0.4 s is 4 0.371 0.332 0.039 -10.5 s == 4 0.932 0.922 0.010 -1.1 s > 4 0.936 0.927 0.009 -1.0 s is -1001 0.370 0.330 0.040 -10.8 s == -1001 0.932 0.923 0.009 -1.0 s > -1001 0.935 0.925 0.010 -1.1 s is 34.7 0.368 0.325 0.043 -11.7 s == 34.7 1.110 1.110 0.000 0.0 s > 34.7 1.110 1.110 0.000 0.0 s is 'a b c' 0.370 0.325 0.045 -12.2 s == 'a b c' 0.963 0.948 0.015 -1.6 s > 'a b c' 0.961 0.949 0.012 -1.2 s is True 0.529 0.490 0.039 -7.4 s == True 1.110 1.110 0.000 0.0 s > True 1.120 1.110 0.010 -0.9 s = -1000 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.371 0.326 0.045 -12.1 s == 'abc' 1.200 1.190 0.010 -0.8 s > 'abc' 1.200 1.190 0.010 -0.8 s is 4 0.349 0.350 -0.001 0.3 s == 4 0.347 0.353 -0.006 1.7 s > 4 0.349 0.347 0.002 -0.6 s is -1001 0.348 0.352 -0.004 1.1 s == -1001 0.349 0.352 -0.003 0.9 s > -1001 0.346 0.348 -0.002 0.6 s is 34.7 0.366 0.326 0.040 -10.9 s == 34.7 0.769 0.771 -0.002 0.3 s > 34.7 0.766 0.777 -0.011 1.4 s is 'a b c' 0.367 0.328 0.039 -10.6 s == 'a b c' 1.210 1.190 0.020 -1.7 s > 'a b c' 1.200 1.190 0.010 -0.8 s is True 0.536 0.490 0.046 -8.6 s == True 0.887 0.887 0.000 0.0 s > True 0.890 0.892 -0.002 0.2 s = 34.2 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.369 0.327 0.042 -11.4 s == 'abc' 1.630 1.620 0.010 -0.6 s > 'abc' 1.640 1.620 0.020 -1.2 s is 4 0.372 0.332 0.040 -10.8 s == 4 0.791 0.795 -0.004 0.5 s > 4 0.797 0.798 -0.001 0.1 s is -1001 0.375 0.331 0.044 -11.7 s == -1001 0.792 0.792 0.000 0.0 s > -1001 0.790 0.791 -0.001 0.1 s is 34.7 0.367 0.482 -0.115 31.3 s == 34.7 1.080 0.536 0.544 -50.4 s > 34.7 0.560 0.621 -0.061 10.9 s is 'a b c' 0.387 0.337 0.050 -12.9 s == 'a b c' 1.760 1.710 0.050 -2.8 s > 'a b c' 1.710 1.680 0.030 -1.8 s is True 0.614 0.509 0.105 -17.1 s == True 1.050 1.020 0.030 -2.9 s > True 1.060 1.020 0.040 -3.8 s = 'a b c' operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.379 0.345 0.034 -9.0 s == 'abc' 0.542 0.494 0.048 -8.9 s > 'abc' 0.586 0.593 -0.007 1.2 s is 4 0.430 0.344 0.086 -20.0 s == 4 1.260 1.230 0.030 -2.4 s > 4 1.370 1.230 0.140 -10.2 s is -1001 0.431 0.372 0.059 -13.7 s == -1001 1.250 1.640 -0.390 31.2 s > -1001 1.240 1.260 -0.020 1.6 s is 34.7 0.383 0.337 0.046 -12.0 s == 34.7 1.770 1.680 0.090 -5.1 s > 34.7 1.670 1.660 0.010 -0.6 s is 'a b c' 0.423 0.376 0.047 -11.1 s == 'a b c' 0.506 0.510 -0.004 0.8 s > 'a b c' 0.517 0.564 -0.047 9.1 s is True 0.550 0.514 0.036 -6.5 s == True 1.470 1.640 -0.170 11.6 s > True 1.450 1.430 0.020 -1.4 s = object() operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.389 0.379 0.010 -2.6 s == 'abc' 1.220 1.370 -0.150 12.3 s > 'abc' 1.220 2.600 -1.380 113.1 s is 4 0.427 0.349 0.078 -18.3 s == 4 1.080 1.620 -0.540 50.0 s > 4 1.060 1.070 -0.010 0.9 s is -1001 0.437 0.343 0.094 -21.5 s == -1001 1.070 1.130 -0.060 5.6 s > -1001 1.060 1.090 -0.030 2.8 s is 34.7 0.419 0.338 0.081 -19.3 s == 34.7 1.710 1.520 0.190 -11.1 s > 34.7 1.520 1.540 -0.020 1.3 s is 'a b c' 0.380 0.347 0.033 -8.7 s == 'a b c' 2.020 1.210 0.810 -40.1 s > 'a b c' 1.260 1.210 0.050 -4.0 s is True 0.622 0.515 0.107 -17.2 s == True 1.220 1.220 0.000 0.0 s > True 1.210 1.210 0.000 0.0 s = [] operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.369 0.326 0.043 -11.7 s == 'abc' 1.220 1.200 0.020 -1.6 s > 'abc' 1.220 1.200 0.020 -1.6 s is 4 0.372 0.332 0.040 -10.8 s == 4 1.160 1.150 0.010 -0.9 s > 4 1.150 1.150 0.000 0.0 s is -1001 0.371 0.334 0.037 -10.0 s == -1001 1.150 1.140 0.010 -0.9 s > -1001 1.150 1.150 0.000 0.0 s is 34.7 0.368 0.326 0.042 -11.4 s == 34.7 1.500 1.480 0.020 -1.3 s > 34.7 1.490 1.490 0.000 0.0 s is 'a b c' 0.366 0.325 0.041 -11.2 s == 'a b c' 1.220 1.200 0.020 -1.6 s > 'a b c' 1.220 1.200 0.020 -1.6 s is True 0.531 0.484 0.047 -8.9 s == True 1.360 1.350 0.010 -0.7 s > True 1.350 1.350 0.000 0.0 I fully expected that the "is" tests would be faster and without question the "==" and ">" tests would be slower. I was quite surprised that this wasn't always the case. The above tests were run on an 800MHz Powerbook G4 running Mac OSX 10.2.8. I don't have immediate access in Intel hardware, though I'll try to run these tests on cygwin this week. I'd be happy to be shown that my shell script isn't measuring what I think it's measuring as well. Skip ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-20 12:27 Message: Logged In: YES user_id=80475 Even "is" and "is not" are not helped by more than a couple of cycles. This fragment essentially inlines part of code for cmp_outcome(). Only the function call is saved. It does slow down other code paths by introducing an unpredictable branch. If the inlining were considered important, then the whole of cmp_outcome() should be inlined. Then, all comparisons save a single call/return pair. The cost is further increasing the size of the eval loop. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-20 11:45 Message: Logged In: YES user_id=31435 Well, there's little question that this will speed "is" and "is not", but it also slows all other cases by the cost of the switch-and-branch to determine that they're not the favored cases. So why should we believe that speeding "is" and "is not" is more important than slowing other cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 From noreply at sourceforge.net Sun Mar 21 10:25:00 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 10:25:04 2004 Subject: [Patches] [ python-Patches-920509 ] Patch to enable profiling of C functions called from python Message-ID: Patches item #920509, was opened at 2004-03-21 10:25 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch to enable profiling of C functions called from python Initial Comment: This patch enables the profiling of C extension functions (and builtins) called via the CALL_FUNCTION opcode. This also includes a modification to lib/profile.py to handle the new call type. On my Powerbook 12" 867Mhz it incurs a 0.5% performance hit in the interpreter when profiling is not enabled. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 From noreply at sourceforge.net Sun Mar 21 10:26:56 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 10:27:01 2004 Subject: [Patches] [ python-Patches-855999 ] Updates to the .spec file. Message-ID: Patches item #855999, was opened at 2003-12-08 02:19 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=855999&group_id=5470 Category: Build Group: Python 2.3 >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Sean Reifschneider (jafo) Assigned to: Nobody/Anonymous (nobody) Summary: Updates to the .spec file. Initial Comment: I'm attaching a patch to the Misc/python-2.3.spec file, here's the changelog: - Adding some build prereqs suggested by Darren Steven and Michael P. Soulier. - Re-locating the code to change /usr/local/bin/python references to after the docs are installed. This version of the spec file seems to work with the Red Hat Fedora release. Sean ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-21 16:26 Message: Logged In: YES user_id=21627 I'm closing this patch as outdated, as it appears to be superceded by 892673. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=855999&group_id=5470 From noreply at sourceforge.net Sun Mar 21 10:28:20 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 10:28:28 2004 Subject: [Patches] [ python-Patches-920509 ] Patch to enable profiling of C functions called from python Message-ID: Patches item #920509, was opened at 2004-03-21 10:25 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch to enable profiling of C functions called from python Initial Comment: This patch enables the profiling of C extension functions (and builtins) called via the CALL_FUNCTION opcode. This also includes a modification to lib/profile.py to handle the new call type. On my Powerbook 12" 867Mhz it incurs a 0.5% performance hit in the interpreter when profiling is not enabled. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-21 10:28 Message: Logged In: YES user_id=430343 Note: this patch is fully backwards with profiler use - the data file is not changed in any way. Also, any 3rd party profilers which would handle unknown event types will still continue to work (the standard python profiler isn't one of those, so the patch includes a fix for that as well). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 From noreply at sourceforge.net Sun Mar 21 10:40:38 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 10:40:47 2004 Subject: [Patches] [ python-Patches-853488 ] Tix hlist missing entry_configure and entry_cget methods Message-ID: Patches item #853488, was opened at 2003-12-03 20:03 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=853488&group_id=5470 Category: Tkinter Group: Python 2.3 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Gene Cash (gcash) Assigned to: Martin v. L?wis (loewis) Summary: Tix hlist missing entry_configure and entry_cget methods Initial Comment: The Tix hlist class doesn't have entry_configure and entry_cget methods to implement "entrycget" and "entryconfigure" as listed in the HList(n) manpage. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-21 16:40 Message: Logged In: YES user_id=21627 Thanks for the patch. Applied as Tix.py 1.18, NEWS 1.953. However, in consistency with Tkinter, I have changed the method names to entryconfigure and entrycget. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=853488&group_id=5470 From noreply at sourceforge.net Sun Mar 21 11:59:14 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 11:59:24 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by jiwon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Jiwon Seo (jiwon) Date: 2004-03-22 01:59 Message: Logged In: YES user_id=595483 Hi, quiver. I don't think we can easily go around this problem if we have to capture iterators in generator expression. If you run following, you'll know what I mean. >>> a = iter("abcd") >>> b = iter("abcd") >>> [(x, y) for x in a for y in b] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] I think one possible solution could be, instead of passing evaluations of iterators in generator expression, make a list from iterator and, then pass it as argument. For instance, g = (x for x in iter("abcd")) will be equivalent to, def __gen(_[1]): for x in _[1]: yield x g = __gen(list(iter("abcd"))) # see 'list' - instead of g = __gen(iter("abcd")) . I'm not sure if I'm in a position to decide to do that way or not. If the current reviewer (rhettinger) approves it, I'll do that way. Or else, I think I will post it on the mailing list. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-03-19 22:37 Message: Logged In: YES user_id=671362 Hi, Jiwon. This is another bug candidate. If you use genexp with iterators, it doesn't work well. # test Perky's previous post using iterators >>> list((x,y) for x in iter('abcd') for y in iter('abcd')) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Thanks. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 17:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sun Mar 21 12:28:31 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 12:28:38 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 07:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-21 12:28 Message: Logged In: YES user_id=80475 You need a solution other than list(). A key to the success of genexps is to never to eat memory by expanding an iterator into a container. For behavior questions, your reference point is the list comprehension. list(genexp) should always produce the same result as a list comprehension. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-21 11:59 Message: Logged In: YES user_id=595483 Hi, quiver. I don't think we can easily go around this problem if we have to capture iterators in generator expression. If you run following, you'll know what I mean. >>> a = iter("abcd") >>> b = iter("abcd") >>> [(x, y) for x in a for y in b] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] I think one possible solution could be, instead of passing evaluations of iterators in generator expression, make a list from iterator and, then pass it as argument. For instance, g = (x for x in iter("abcd")) will be equivalent to, def __gen(_[1]): for x in _[1]: yield x g = __gen(list(iter("abcd"))) # see 'list' - instead of g = __gen(iter("abcd")) . I'm not sure if I'm in a position to decide to do that way or not. If the current reviewer (rhettinger) approves it, I'll do that way. Or else, I think I will post it on the mailing list. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-03-19 08:37 Message: Logged In: YES user_id=671362 Hi, Jiwon. This is another bug candidate. If you use genexp with iterators, it doesn't work well. # test Perky's previous post using iterators >>> list((x,y) for x in iter('abcd') for y in iter('abcd')) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Thanks. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 03:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 01:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 01:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 00:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-16 11:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-15 23:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-13 21:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-13 16:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-23 11:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 08:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 05:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 01:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 06:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 04:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 07:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 07:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 06:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 03:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 00:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-29 22:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 06:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 02:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-26 10:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 01:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-25 18:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-21 13:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 09:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 06:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-11 22:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 20:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 00:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-07 23:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 07:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sun Mar 21 12:38:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 12:38:31 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:38 Message: Logged In: YES user_id=55188 Agreed. I'd prefer the current implementation's behavor rather than defined in PEP289. Yes, It's incompatible with list comprehensions but generator expressions are already quite different enough from it. :) How about to change PEP289's genexpr semantics to this? g = (x**2 for x in range(10)) print g.next() is equivalent to: def __gen(_i1): for x in _i1: yield x**2 g = __gen(range(10)) print g.next() ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-22 02:28 Message: Logged In: YES user_id=80475 You need a solution other than list(). A key to the success of genexps is to never to eat memory by expanding an iterator into a container. For behavior questions, your reference point is the list comprehension. list(genexp) should always produce the same result as a list comprehension. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-22 01:59 Message: Logged In: YES user_id=595483 Hi, quiver. I don't think we can easily go around this problem if we have to capture iterators in generator expression. If you run following, you'll know what I mean. >>> a = iter("abcd") >>> b = iter("abcd") >>> [(x, y) for x in a for y in b] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] I think one possible solution could be, instead of passing evaluations of iterators in generator expression, make a list from iterator and, then pass it as argument. For instance, g = (x for x in iter("abcd")) will be equivalent to, def __gen(_[1]): for x in _[1]: yield x g = __gen(list(iter("abcd"))) # see 'list' - instead of g = __gen(iter("abcd")) . I'm not sure if I'm in a position to decide to do that way or not. If the current reviewer (rhettinger) approves it, I'll do that way. Or else, I think I will post it on the mailing list. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-03-19 22:37 Message: Logged In: YES user_id=671362 Hi, Jiwon. This is another bug candidate. If you use genexp with iterators, it doesn't work well. # test Perky's previous post using iterators >>> list((x,y) for x in iter('abcd') for y in iter('abcd')) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Thanks. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 17:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sun Mar 21 12:42:10 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 12:42:18 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:42 Message: Logged In: YES user_id=55188 Aah. But I'm not insisting on that because it's incompatible even with plain nested for-loops. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:38 Message: Logged In: YES user_id=55188 Agreed. I'd prefer the current implementation's behavor rather than defined in PEP289. Yes, It's incompatible with list comprehensions but generator expressions are already quite different enough from it. :) How about to change PEP289's genexpr semantics to this? g = (x**2 for x in range(10)) print g.next() is equivalent to: def __gen(_i1): for x in _i1: yield x**2 g = __gen(range(10)) print g.next() ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-22 02:28 Message: Logged In: YES user_id=80475 You need a solution other than list(). A key to the success of genexps is to never to eat memory by expanding an iterator into a container. For behavior questions, your reference point is the list comprehension. list(genexp) should always produce the same result as a list comprehension. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-22 01:59 Message: Logged In: YES user_id=595483 Hi, quiver. I don't think we can easily go around this problem if we have to capture iterators in generator expression. If you run following, you'll know what I mean. >>> a = iter("abcd") >>> b = iter("abcd") >>> [(x, y) for x in a for y in b] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] I think one possible solution could be, instead of passing evaluations of iterators in generator expression, make a list from iterator and, then pass it as argument. For instance, g = (x for x in iter("abcd")) will be equivalent to, def __gen(_[1]): for x in _[1]: yield x g = __gen(list(iter("abcd"))) # see 'list' - instead of g = __gen(iter("abcd")) . I'm not sure if I'm in a position to decide to do that way or not. If the current reviewer (rhettinger) approves it, I'll do that way. Or else, I think I will post it on the mailing list. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-03-19 22:37 Message: Logged In: YES user_id=671362 Hi, Jiwon. This is another bug candidate. If you use genexp with iterators, it doesn't work well. # test Perky's previous post using iterators >>> list((x,y) for x in iter('abcd') for y in iter('abcd')) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Thanks. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 17:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sun Mar 21 12:43:03 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 12:43:11 2004 Subject: [Patches] [ python-Patches-871657 ] math.sqrt EDOM handling for FreeBSD, OpenBSD Message-ID: Patches item #871657, was opened at 2004-01-06 22:40 Message generated for change (Settings changed) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=871657&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Hye-Shik Chang (perky) >Assigned to: Hye-Shik Chang (perky) Summary: math.sqrt EDOM handling for FreeBSD, OpenBSD Initial Comment: math.sqrt(-1) doesn't raise ValueError currently. The attached patch may fix the problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=871657&group_id=5470 From noreply at sourceforge.net Sun Mar 21 12:55:17 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 12:55:34 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by perky You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:55 Message: Logged In: YES user_id=55188 This inconsistency may not be fixed by list(iter()) workaround, either. >>> def counter(): ... counter.count += 1 ... return range(counter.count) ... >>> counter.count = 0 >>> [y for x in 'abc' for y in counter()] [0, 0, 1, 0, 1, 2] >>> counter.count = 0 >>> list(y for x in 'abc' for y in counter()) [0, 0, 0] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:42 Message: Logged In: YES user_id=55188 Aah. But I'm not insisting on that because it's incompatible even with plain nested for-loops. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:38 Message: Logged In: YES user_id=55188 Agreed. I'd prefer the current implementation's behavor rather than defined in PEP289. Yes, It's incompatible with list comprehensions but generator expressions are already quite different enough from it. :) How about to change PEP289's genexpr semantics to this? g = (x**2 for x in range(10)) print g.next() is equivalent to: def __gen(_i1): for x in _i1: yield x**2 g = __gen(range(10)) print g.next() ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-22 02:28 Message: Logged In: YES user_id=80475 You need a solution other than list(). A key to the success of genexps is to never to eat memory by expanding an iterator into a container. For behavior questions, your reference point is the list comprehension. list(genexp) should always produce the same result as a list comprehension. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-22 01:59 Message: Logged In: YES user_id=595483 Hi, quiver. I don't think we can easily go around this problem if we have to capture iterators in generator expression. If you run following, you'll know what I mean. >>> a = iter("abcd") >>> b = iter("abcd") >>> [(x, y) for x in a for y in b] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] I think one possible solution could be, instead of passing evaluations of iterators in generator expression, make a list from iterator and, then pass it as argument. For instance, g = (x for x in iter("abcd")) will be equivalent to, def __gen(_[1]): for x in _[1]: yield x g = __gen(list(iter("abcd"))) # see 'list' - instead of g = __gen(iter("abcd")) . I'm not sure if I'm in a position to decide to do that way or not. If the current reviewer (rhettinger) approves it, I'll do that way. Or else, I think I will post it on the mailing list. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-03-19 22:37 Message: Logged In: YES user_id=671362 Hi, Jiwon. This is another bug candidate. If you use genexp with iterators, it doesn't work well. # test Perky's previous post using iterators >>> list((x,y) for x in iter('abcd') for y in iter('abcd')) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Thanks. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 17:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:06:06 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:06:11 2004 Subject: [Patches] [ python-Patches-850977 ] Modify Setup.py to Detect Tcl/Tk on BSD Message-ID: Patches item #850977, was opened at 2003-11-28 19:19 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=850977&group_id=5470 Category: Build >Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Kurt B. Kaiser (kbk) Assigned to: A.M. Kuchling (akuchling) Summary: Modify Setup.py to Detect Tcl/Tk on BSD Initial Comment: FreeBSD and OpenBSD set up libs as .../libtcl83 but includes as .../include/tcl8.3 Patch allows detection on OpenBSD w/o the use of Modules/Setup.local as is currently done in the port, also allows easier Python development on that platform (build w/o using ports). This is just an enhancement of the Debian detection method. Tested on OpenBSD and Linux. Assigning to Andrew for review, he's the last to work on this section of the code. Backport 2.3 candidate? ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2004-03-21 14:06 Message: Logged In: YES user_id=11375 Looks reasonable; applying to 2.4. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=850977&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:10:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:10:31 2004 Subject: [Patches] [ python-Patches-905863 ] support CVS version of tcl/tk ("8.5") Message-ID: Patches item #905863, was opened at 2004-02-27 08:56 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=905863&group_id=5470 Category: Tkinter Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Jeff Epler (jepler) >Assigned to: A.M. Kuchling (akuchling) Summary: support CVS version of tcl/tk ("8.5") Initial Comment: This patch makes setup.py search for a version of tcl/tk calling itself "8.5". This is the version number in the current CVS version of tcl/tk ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2004-03-21 14:10 Message: Logged In: YES user_id=11375 Seems reasonable; applied to HEAD. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=905863&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:17:57 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:18:02 2004 Subject: [Patches] [ python-Patches-918212 ] XHTML support for webchecker.py Message-ID: Patches item #918212, was opened at 2004-03-17 13:40 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918212&group_id=5470 Category: Demos and tools Group: Python 2.3 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Chris Herborth (docoverlord) >Assigned to: A.M. Kuchling (akuchling) Summary: XHTML support for webchecker.py Initial Comment: Here's a small patch that adds support for the 'id' attribute ('name' is deprecated in XHTML) to Tools/webchecker. README and webchecker.py are affected. Tested over approx. 26,000 lines of XHTML. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2004-03-21 14:17 Message: Logged In: YES user_id=11375 Looks good; applied to HEAD. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918212&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:22:14 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:22:20 2004 Subject: [Patches] [ python-Patches-510695 ] cycle profiler for VM opcodes Message-ID: Patches item #510695, was opened at 2002-01-30 08:21 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=510695&group_id=5470 Category: Core (C code) Group: None Status: Open >Resolution: Out of Date Priority: 1 Submitted By: Jeremy Hylton (jhylton) Assigned to: Jeremy Hylton (jhylton) Summary: cycle profiler for VM opcodes Initial Comment: This is just some code I'm noodling around with. It counts the number of cycles each Python VM opcode takes to execute, using the Pentium timestamp counter. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:22 Message: Logged In: YES user_id=31435 Changed resolution from Accepted to Out-of-Date -- several parts of this patch fail to apply now. Leaving assigned to Jeremy. Still seems like a fine idea to me! ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-09-20 07:40 Message: Logged In: YES user_id=21627 OTOH, the patch is prepared for temporary usage, by means of a configure option. Since nobody has been objecting strongly, I'll accept this patch. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2002-11-12 08:26 Message: Logged In: YES user_id=4771 I think that you should try and convince people that it is a generally useful information to have, and that the fact that it is highly non-portable does not hurt. Right now it looks more like a change that a core developer would temporarily want to do to tune the VM. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2002-10-09 11:16 Message: Logged In: YES user_id=31392 sure ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2002-10-07 17:12 Message: Logged In: YES user_id=21627 Is this still of relevance? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=510695&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:23:20 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:23:26 2004 Subject: [Patches] [ python-Patches-510695 ] cycle profiler for VM opcodes Message-ID: Patches item #510695, was opened at 2002-01-30 08:21 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=510695&group_id=5470 Category: Core (C code) Group: None Status: Open Resolution: Out of Date Priority: 1 Submitted By: Jeremy Hylton (jhylton) Assigned to: Jeremy Hylton (jhylton) Summary: cycle profiler for VM opcodes Initial Comment: This is just some code I'm noodling around with. It counts the number of cycles each Python VM opcode takes to execute, using the Pentium timestamp counter. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:23 Message: Logged In: YES user_id=31435 Changed resolution from Accepted to Out-of-Date -- several parts of this patch fail to apply now. Leaving assigned to Jeremy. Still seems like a fine idea to me! ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:22 Message: Logged In: YES user_id=31435 Changed resolution from Accepted to Out-of-Date -- several parts of this patch fail to apply now. Leaving assigned to Jeremy. Still seems like a fine idea to me! ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-09-20 07:40 Message: Logged In: YES user_id=21627 OTOH, the patch is prepared for temporary usage, by means of a configure option. Since nobody has been objecting strongly, I'll accept this patch. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2002-11-12 08:26 Message: Logged In: YES user_id=4771 I think that you should try and convince people that it is a generally useful information to have, and that the fact that it is highly non-portable does not hurt. Right now it looks more like a change that a core developer would temporarily want to do to tune the VM. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2002-10-09 11:16 Message: Logged In: YES user_id=31392 sure ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2002-10-07 17:12 Message: Logged In: YES user_id=21627 Is this still of relevance? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=510695&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:24:23 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:24:30 2004 Subject: [Patches] [ python-Patches-510695 ] cycle profiler for VM opcodes Message-ID: Patches item #510695, was opened at 2002-01-30 08:21 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=510695&group_id=5470 Category: Core (C code) Group: None Status: Open Resolution: Out of Date Priority: 1 Submitted By: Jeremy Hylton (jhylton) Assigned to: Jeremy Hylton (jhylton) Summary: cycle profiler for VM opcodes Initial Comment: This is just some code I'm noodling around with. It counts the number of cycles each Python VM opcode takes to execute, using the Pentium timestamp counter. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:24 Message: Logged In: YES user_id=31435 Changed resolution from Accepted to Out-of-Date -- several parts of this patch fail to apply now. Leaving assigned to Jeremy. Still seems like a fine idea to me! ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:23 Message: Logged In: YES user_id=31435 Changed resolution from Accepted to Out-of-Date -- several parts of this patch fail to apply now. Leaving assigned to Jeremy. Still seems like a fine idea to me! ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:22 Message: Logged In: YES user_id=31435 Changed resolution from Accepted to Out-of-Date -- several parts of this patch fail to apply now. Leaving assigned to Jeremy. Still seems like a fine idea to me! ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-09-20 07:40 Message: Logged In: YES user_id=21627 OTOH, the patch is prepared for temporary usage, by means of a configure option. Since nobody has been objecting strongly, I'll accept this patch. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2002-11-12 08:26 Message: Logged In: YES user_id=4771 I think that you should try and convince people that it is a generally useful information to have, and that the fact that it is highly non-portable does not hurt. Right now it looks more like a change that a core developer would temporarily want to do to tune the VM. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2002-10-09 11:16 Message: Logged In: YES user_id=31392 sure ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2002-10-07 17:12 Message: Logged In: YES user_id=21627 Is this still of relevance? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=510695&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:27:18 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:27:31 2004 Subject: [Patches] [ python-Patches-517256 ] poor performance in xmlrpc response Message-ID: Patches item #517256, was opened at 2002-02-13 18:48 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=517256&group_id=5470 Category: Library (Lib) Group: Python 2.1.2 Status: Open Resolution: Accepted Priority: 5 Submitted By: James Rucker (jamesrucker) Assigned to: Fredrik Lundh (effbot) Summary: poor performance in xmlrpc response Initial Comment: xmlrpclib.Transport.parse_response() (called from xmlrpclib.Transport.request()) is exhibiting poor performance - approx. 10x slower than expected. I investigated based on using a simple app that sent a msg to a server, where all the server did was return the message back to the caller. From profiling, it became clear that the return trip was taken 10x the time consumed by the client->server trip, and that the time was spent getting things across the wire. parse_response() reads from a file object created via socket.makefile(), and as a result exhibits performance that is about an order of magnitude worse than what it would be if socket.recv() were used on the socket. The patch provided uses socket.recv() when possible, to improve performance. The patch provided is against revision 1.15. Its use provides performance for the return trip that is more or less equivalent to that of the forward trip. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:27 Message: Logged In: YES user_id=31435 It's been 2 years since this report was updated. Any chance of resolving it? If not, please close it. ---------------------------------------------------------------------- Comment By: James Rucker (jamesrucker) Date: 2002-03-19 14:47 Message: Logged In: YES user_id=351540 HTTPConnection.getresponse() will close the socket and set self.sock to null after instantiating response_class (by default, this is HTTPResponse; note that HTTPResponse does a makefile() and stores the result in self.fp) iff the newly created response class instance's 'will_close' attribute is true. My server is setting the Keep-alive header with a value of 1 (it is based on xmlrpcserver.py), which causes will_close to evaluate to false. In your case, I'm presuming that will_close is being evaluated as false and thus the socket (accessed via h._conn.sock) has been set to . Note that when I removed the Keep-alive header, I witness the behaviour you're seeing. Thus, it seems that as it stands, the beneift of the change will only be realized if Keep-alive is set or HTTP/1.1 is used (and Keep-alive is either not specified or is set to non-zero). The following from httplib.py shows and explains how 'will_close' will be set (from httplib.py): conn = self.msg.getheader('connection') if conn: conn = conn.lower() # a "Connection: close" will always close the connection. if we # don't see that and this is not HTTP/1.1, then the connection will # close unless we see a Keep-Alive header. self.will_close = conn.find('close') != -1 or ( self.version != 11 and not self.msg.getheader('keep-alive') ) else: # for HTTP/1.1, the connection will always remain open # otherwise, it will remain open IFF we see a Keep-Alive header self.will_close = self.version != 11 and not self.msg.getheader('keep-alive') ---------------------------------------------------------------------- Comment By: Fredrik Lundh (effbot) Date: 2002-03-19 04:24 Message: Logged In: YES user_id=38376 What server did you use? In all my test setups, h._conn.sock is None at the time parse_response is called... ---------------------------------------------------------------------- Comment By: James Rucker (jamesrucker) Date: 2002-03-17 11:13 Message: Logged In: YES user_id=351540 The problem was discovered under FreeBSD 4.4. ---------------------------------------------------------------------- Comment By: Fredrik Lundh (effbot) Date: 2002-03-17 08:30 Message: Logged In: YES user_id=38376 James, what platform(s) did you use? I'm not sure changing the parse_response() interface is a good idea, but if this is a Windows-only problem, there may be a slightly cleaner way to get the same end result. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2002-03-01 11:14 Message: Logged In: YES user_id=6380 My guess makefile() isn't buffering properly. This has been a long-standing problem on Windows; I'm not sure if it's an issue on Unix. ---------------------------------------------------------------------- Comment By: Fredrik Lundh (effbot) Date: 2002-03-01 09:34 Message: Logged In: YES user_id=38376 looks fine to me. I'll merge it with SLAB changes, and will check it into the 2.3 codebase asap. (we probably should try to figure out why makefile causes a 10x slowdown too -- xmlrpclib isn't exactly the only client library reading from a buffered socket) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2002-02-28 18:23 Message: Logged In: YES user_id=6380 Fredrik, does this look OK to you? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=517256&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:31:38 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:31:43 2004 Subject: [Patches] [ python-Patches-521478 ] mailbox / fromline matching Message-ID: Patches item #521478, was opened at 2002-02-22 09:54 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=521478&group_id=5470 Category: Library (Lib) Group: Python 2.2.x Status: Open Resolution: Rejected Priority: 5 Submitted By: Camiel Dobbelaar (camield) Assigned to: Barry A. Warsaw (bwarsaw) Summary: mailbox / fromline matching Initial Comment: mailbox.py does not parse this 'From' line correctly: >From camield@sentia.nl Mon Apr 23 18:22:28 2001 +0200 ^^^^^ This is because of the trailing timezone information, that the regex does not account for. Also, 'From' should match at the beginning of the line. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-21 14:31 Message: Logged In: YES user_id=31435 Since the Monday in question happened over 2 years ago, the answer to Michael's question is apparently "no" . Barry, we're stretching the conventional meaning of "asap" here -- can you close this one way or t'other now? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2002-03-16 11:43 Message: Logged In: YES user_id=6656 Anything going to happen here by Monday? ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2002-03-02 11:47 Message: Logged In: YES user_id=12800 Re-opening and assigning to myself. I'll take a look at your patches asap. ---------------------------------------------------------------------- Comment By: Camiel Dobbelaar (camield) Date: 2002-03-02 09:34 Message: Logged In: YES user_id=466784 PortableUnixMailbox is not that useful, because it only matches '^From '. From-quoting is an even bigger mess then From-headerlines, so that does not really help. I submit a new diff that matches '\n\nFrom ' or 'From ', which makes PortableUnixMailbox useful for my purposes. It is not that intrusive as the comment in the mailbox.py suggests. ---------------------------------------------------------------------- Comment By: Barry A. Warsaw (bwarsaw) Date: 2002-03-01 16:42 Message: Logged In: YES user_id=12800 IMO, Jamie Zawinski (author of the original mail/news reader in Netscape among other accomplishments), wrote the definitive answer on From_ http://home.netscape.com/eng/mozilla/2.0/relnotes/demo/content-length.html As far as Python's support for this in the mailbox module, for backwards compatibility, the UnixMailbox class has a strict-ish interpretation of the From_ delimiter, which I think should not change. It also has a class called PortableUnixMailbox which recognizes delimiters as specified in JWZ's document. Personally, if I was trolling over a real world mbox file I'd only use PortableUnixMailbox (as long as non-delimiter From_ lines were properly escaped -- I have some code in Mailman which tries to intelligently "fix" non-escaped mbox files). I agree with the Rejected resolution. ---------------------------------------------------------------------- Comment By: Camiel Dobbelaar (camield) Date: 2002-03-01 06:34 Message: Logged In: YES user_id=466784 I have tracked this down to Pine, the mailreader. In imap/src/c-client/mail.c, it has this flag: static int notimezones = NIL; /* write timezones in "From " header */ (so timezones are written in the "From" lines by default) I also found the following comment in imap/docs/FAQ in the Pine distribution: """ So, good mail reading software only considers a line to be a "From " line if it follows the actual specification for a "From " line. This means, among other things, that the day of week is fixed-format: "May 14", but "May 7" (note the extra space) as opposed to "May 7". ctime() format for the date is the most common, although POSIX also allows a numeric timezone after the year. """ While I don't consider Pine to be the ultimate mailreader, its heritage may warrant that the 'From ' lines it creates are considered 'standard'. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2002-02-28 17:37 Message: Logged In: YES user_id=6380 That From line is simply illegal, or at least nonstandard. If your system uses this nonstandard format, you can extend the mailbox parser by overriding the ._isrealfromline method. The pattern doesn't need ^ because match() is used, which only matches at the start of the line. Rejected. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=521478&group_id=5470 From noreply at sourceforge.net Sun Mar 21 14:34:23 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 21 14:34:34 2004 Subject: [Patches] [ python-Patches-545480 ] Examples for urllib2 Message-ID: Patches item #545480, was opened at 2002-04-18 00:13 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=545480&group_id=5470 Category: Documentation Group: None >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Sean Reifschneider (jafo) Assigned to: Jeremy Hylton (jhylton) Summary: Examples for urllib2 Initial Comment: An associate who's learning Python recently complained about a lack of examples for urllib2. As a starting point, I'd like to submit the following: This example gets the python.org main page and displays the first 100 bytes of it: >>> import urllib2 >>> url = urllib2.urlopen('http://www.python.org/') >>> print url.read()[:100] weighted result 1.8 microsec lost Of course, this can't be done exactly or even inexactly, but it shows that the percentages can't be considered out of the context of dynamic usage frequency, aggregations of all the operators, and real time. If something like this patch needs to go in, consider making the branches predictable: slow_compare: if (oparg == PyCmp_IS) { x = (v == w) ? Py_True : Py_False; Py_INCREF(x); } else if (oparg == PyCmp_IS_NOT) { x = (v != w) ? Py_True : Py_False; Py_INCREF(x); } else x = cmp_outcome(oparg, v, w); Also, when it comes to micro-optimizations that are compiler sensitive, the Intel timing tests should be built with the compiler actually used to build the distribution (no sense convincing ourselves of an optimization that doesn't occur on the real distribution). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-23 00:15 Message: Logged In: YES user_id=31435 When you introduce a new branch, and time it in isolation, HW may have enough resource to optimize for both branch targets simultaneously. Run a ton of other stuff too, though, and then it can start to lose. Still, for detailed answers about anything at this level, you need to use a HW simulator -- modern processors are intractably complex, and the user- visible programming model supplied by Pentium in particular is multiple layers removed from bottom-line reality now, so much so that Intel doesn't even try to supply "instruction timings" anymore (they depend in complex ways on the internal states of resources that aren't visible in the programming model). ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2004-03-22 16:53 Message: Logged In: YES user_id=44345 I reran the test on a Linux system today and got similar results. I'm pasting them here mostly as documentation. I'm still a bit confused why the == and > tests should show improvement, but they often do on both platforms. Any ideas? Looking at the assembly code generated GCC inserts basically the same four instructions on both the Intel and PowerPC platforms: cmpl $8, -40(%ebp) je .L580 cmpl $9, -40(%ebp) je .L583 on Intel or cmpwi cr7,r24,8 beq- cr7,L622 cmpwi cr7,r24,9 beq- cr7,L625 on PowerPC. I also tried pystone. I see performance hits on both Linux and Mac OSX: Fastest of ten runs patched unpatched Linux 37878.8 38167.9 Mac OSX 13888.9 14124.3 Oh well... It was a thought. Test output on Linux: s = 'abc' operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.116 0.103 0.013 -11.2 s == 'abc' 0.145 0.141 0.004 -2.8 s > 'abc' 0.140 0.142 -0.002 1.4 s is 4 0.139 0.121 0.018 -12.9 s == 4 0.271 0.293 -0.022 8.1 s > 4 0.276 0.273 0.003 -1.1 s is -1001 0.126 0.120 0.006 -4.8 s == -1001 0.270 0.272 -0.002 0.7 s > -1001 0.282 0.275 0.007 -2.5 s is 34.7 0.133 0.119 0.014 -10.5 s == 34.7 0.352 0.343 0.009 -2.6 s > 34.7 0.340 0.344 -0.004 1.2 s is 'a b c' 0.135 0.118 0.017 -12.6 s == 'a b c' 0.159 0.157 0.002 -1.3 s > 'a b c' 0.200 0.201 -0.001 0.5 s is True 0.177 0.170 0.007 -4.0 s == True 0.316 0.318 -0.002 0.6 s > True 0.321 0.321 0.000 0.0 s = 4 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.143 0.120 0.023 -16.1 s == 'abc' 0.266 0.285 -0.019 7.1 s > 'abc' 0.270 0.276 -0.006 2.2 s is 4 0.175 0.103 0.072 -41.1 s == 4 0.105 0.105 0.000 0.0 s > 4 0.106 0.107 -0.001 0.9 s is -1001 0.119 0.119 0.000 0.0 s == -1001 0.119 0.119 0.000 0.0 s > -1001 0.121 0.178 -0.057 47.1 s is 34.7 0.127 0.129 -0.002 1.6 s == 34.7 0.201 0.195 0.006 -3.0 s > 34.7 0.193 0.197 -0.004 2.1 s is 'a b c' 0.212 0.125 0.087 -41.0 s == 'a b c' 0.268 0.271 -0.003 1.1 s > 'a b c' 0.269 0.276 -0.007 2.6 s is True 0.196 0.160 0.036 -18.4 s == True 0.239 0.258 -0.019 7.9 s > True 0.265 0.237 0.028 -10.6 s = None operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.120 0.109 0.011 -9.2 s == 'abc' 0.203 0.204 -0.001 0.5 s > 'abc' 0.206 0.206 0.000 0.0 s is 4 0.119 0.110 0.009 -7.6 s == 4 0.217 0.214 0.003 -1.4 s > 4 0.214 0.220 -0.006 2.8 s is -1001 0.120 0.107 0.013 -10.8 s == -1001 0.207 0.207 0.000 0.0 s > -1001 0.207 0.214 -0.007 3.4 s is 34.7 0.122 0.112 0.010 -8.2 s == 34.7 0.274 0.270 0.004 -1.5 s > 34.7 0.272 0.271 0.001 -0.4 s is 'a b c' 0.148 0.128 0.020 -13.5 s == 'a b c' 0.240 0.242 -0.002 0.8 s > 'a b c' 0.206 0.210 -0.004 1.9 s is True 0.162 0.153 0.009 -5.6 s == True 0.267 0.262 0.005 -1.9 s > True 0.284 0.258 0.026 -9.2 s = -1000 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.218 0.128 0.090 -41.3 s == 'abc' 0.274 0.275 -0.001 0.4 s > 'abc' 0.264 0.301 -0.037 14.0 s is 4 0.125 0.120 0.005 -4.0 s == 4 0.123 0.122 0.001 -0.8 s > 4 0.119 0.121 -0.002 1.7 s is -1001 0.123 0.123 0.000 0.0 s == -1001 0.132 0.123 0.009 -6.8 s > -1001 0.121 0.121 0.000 0.0 s is 34.7 0.130 0.215 -0.085 65.4 s == 34.7 0.199 0.197 0.002 -1.0 s > 34.7 0.194 0.236 -0.042 21.6 s is 'a b c' 0.158 0.140 0.018 -11.4 s == 'a b c' 0.294 0.293 0.001 -0.3 s > 'a b c' 0.302 0.300 0.002 -0.7 s is True 0.190 0.161 0.029 -15.3 s == True 0.234 0.232 0.002 -0.9 s > True 0.238 0.234 0.004 -1.7 s = 34.2 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.133 0.120 0.013 -9.8 s == 'abc' 0.338 0.330 0.008 -2.4 s > 'abc' 0.350 0.338 0.012 -3.4 s is 4 0.126 0.121 0.005 -4.0 s == 4 0.194 0.197 -0.003 1.5 s > 4 0.193 0.196 -0.003 1.6 s is -1001 0.132 0.120 0.012 -9.1 s == -1001 0.293 0.193 0.100 -34.1 s > -1001 0.196 0.190 0.006 -3.1 s is 34.7 0.117 0.105 0.012 -10.3 s == 34.7 0.153 0.153 0.000 0.0 s > 34.7 0.156 0.155 0.001 -0.6 s is 'a b c' 0.152 0.138 0.014 -9.2 s == 'a b c' 0.360 0.398 -0.038 10.6 s > 'a b c' 0.334 0.354 -0.020 6.0 s is True 0.171 0.174 -0.003 1.8 s == True 0.248 0.254 -0.006 2.4 s > True 0.247 0.244 0.003 -1.2 s = 'a b c' operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.137 0.117 0.020 -14.6 s == 'abc' 0.157 0.158 -0.001 0.6 s > 'abc' 0.204 0.201 0.003 -1.5 s is 4 0.131 0.119 0.012 -9.2 s == 4 0.269 0.272 -0.003 1.1 s > 4 0.277 0.277 0.000 0.0 s is -1001 0.153 0.146 0.007 -4.6 s == -1001 0.299 0.294 0.005 -1.7 s > -1001 0.299 0.302 -0.003 1.0 s is 34.7 0.153 0.146 0.007 -4.6 s == 34.7 0.374 0.368 0.006 -1.6 s > 34.7 0.342 0.336 0.006 -1.8 s is 'a b c' 0.140 0.118 0.022 -15.7 s == 'a b c' 0.150 0.158 -0.008 5.3 s > 'a b c' 0.160 0.156 0.004 -2.5 s is True 0.193 0.194 -0.001 0.5 s == True 0.345 0.338 0.007 -2.0 s > True 0.318 0.319 -0.001 0.3 s = object() operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.158 0.143 0.015 -9.5 s == 'abc' 0.298 0.294 0.004 -1.3 s > 'abc' 0.288 0.292 -0.004 1.4 s is 4 0.129 0.121 0.008 -6.2 s == 4 0.249 0.250 -0.001 0.4 s > 4 0.248 0.249 -0.001 0.4 s is -1001 0.151 0.152 -0.001 0.7 s == -1001 0.271 0.266 0.005 -1.8 s > -1001 0.284 0.271 0.013 -4.6 s is 34.7 0.152 0.140 0.012 -7.9 s == 34.7 0.364 0.385 -0.021 5.8 s > 34.7 0.429 0.392 0.037 -8.6 s is 'a b c' 0.152 0.138 0.014 -9.2 s == 'a b c' 0.300 0.297 0.003 -1.0 s > 'a b c' 0.288 0.285 0.003 -1.0 s is True 0.192 0.184 0.008 -4.2 s == True 0.325 0.329 -0.004 1.2 s > True 0.324 0.322 0.002 -0.6 s = [] operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.126 0.121 0.005 -4.0 s == 'abc' 0.266 0.285 -0.019 7.1 s > 'abc' 0.273 0.271 0.002 -0.7 s is 4 0.125 0.119 0.006 -4.8 s == 4 0.269 0.269 0.000 0.0 s > 4 0.268 0.274 -0.006 2.2 s is -1001 0.133 0.121 0.012 -9.0 s == -1001 0.269 0.291 -0.022 8.2 s > -1001 0.271 0.269 0.002 -0.7 s is 34.7 0.132 0.124 0.008 -6.1 s == 34.7 0.332 0.362 -0.030 9.0 s > 34.7 0.339 0.336 0.003 -0.9 s is 'a b c' 0.125 0.119 0.006 -4.8 s == 'a b c' 0.268 0.291 -0.023 8.6 s > 'a b c' 0.275 0.273 0.002 -0.7 s is True 0.171 0.164 0.007 -4.1 s == True 0.317 0.315 0.002 -0.6 s > True 0.338 0.316 0.022 -6.5 ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2004-03-21 09:33 Message: Logged In: YES user_id=44345 I spent a fair amount of time yesterday refining and running a shell script (attached) to compare the before and after times for various comparisons of simple objects. Here's the output: s = 'abc' operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.375 0.329 0.046 -12.3 s == 'abc' 0.491 0.493 -0.002 0.4 s > 'abc' 0.491 0.493 -0.002 0.4 s is 4 0.375 0.333 0.042 -11.2 s == 4 1.200 1.190 0.010 -0.8 s > 4 1.200 1.190 0.010 -0.8 s is -1001 0.378 0.332 0.046 -12.2 s == -1001 1.200 1.190 0.010 -0.8 s > -1001 1.200 1.180 0.020 -1.7 s is 34.7 0.370 0.325 0.045 -12.2 s == 34.7 1.620 1.590 0.030 -1.9 s > 34.7 1.600 1.590 0.010 -0.6 s is 'a b c' 0.369 0.328 0.041 -11.1 s == 'a b c' 0.475 0.476 -0.001 0.2 s > 'a b c' 0.559 0.563 -0.004 0.7 s is True 0.531 0.491 0.040 -7.5 s == True 1.400 1.390 0.010 -0.7 s > True 1.400 1.380 0.020 -1.4 s = 4 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.369 0.325 0.044 -11.9 s == 'abc' 1.200 1.190 0.010 -0.8 s > 'abc' 1.200 1.190 0.010 -0.8 s is 4 0.353 0.353 0.000 0.0 s == 4 0.352 0.355 -0.003 0.9 s > 4 0.354 0.350 0.004 -1.1 s is -1001 0.347 0.350 -0.003 0.9 s == -1001 0.350 0.353 -0.003 0.9 s > -1001 0.346 0.345 0.001 -0.3 s is 34.7 0.367 0.327 0.040 -10.9 s == 34.7 0.773 0.769 0.004 -0.5 s > 34.7 0.771 0.772 -0.001 0.1 s is 'a b c' 0.370 0.327 0.043 -11.6 s == 'a b c' 1.200 1.190 0.010 -0.8 s > 'a b c' 1.200 1.190 0.010 -0.8 s is True 0.534 0.492 0.042 -7.9 s == True 0.905 0.911 -0.006 0.7 s > True 0.904 0.913 -0.009 1.0 s = None operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.368 0.327 0.041 -11.1 s == 'abc' 0.962 0.950 0.012 -1.2 s > 'abc' 0.959 0.955 0.004 -0.4 s is 4 0.371 0.332 0.039 -10.5 s == 4 0.932 0.922 0.010 -1.1 s > 4 0.936 0.927 0.009 -1.0 s is -1001 0.370 0.330 0.040 -10.8 s == -1001 0.932 0.923 0.009 -1.0 s > -1001 0.935 0.925 0.010 -1.1 s is 34.7 0.368 0.325 0.043 -11.7 s == 34.7 1.110 1.110 0.000 0.0 s > 34.7 1.110 1.110 0.000 0.0 s is 'a b c' 0.370 0.325 0.045 -12.2 s == 'a b c' 0.963 0.948 0.015 -1.6 s > 'a b c' 0.961 0.949 0.012 -1.2 s is True 0.529 0.490 0.039 -7.4 s == True 1.110 1.110 0.000 0.0 s > True 1.120 1.110 0.010 -0.9 s = -1000 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.371 0.326 0.045 -12.1 s == 'abc' 1.200 1.190 0.010 -0.8 s > 'abc' 1.200 1.190 0.010 -0.8 s is 4 0.349 0.350 -0.001 0.3 s == 4 0.347 0.353 -0.006 1.7 s > 4 0.349 0.347 0.002 -0.6 s is -1001 0.348 0.352 -0.004 1.1 s == -1001 0.349 0.352 -0.003 0.9 s > -1001 0.346 0.348 -0.002 0.6 s is 34.7 0.366 0.326 0.040 -10.9 s == 34.7 0.769 0.771 -0.002 0.3 s > 34.7 0.766 0.777 -0.011 1.4 s is 'a b c' 0.367 0.328 0.039 -10.6 s == 'a b c' 1.210 1.190 0.020 -1.7 s > 'a b c' 1.200 1.190 0.010 -0.8 s is True 0.536 0.490 0.046 -8.6 s == True 0.887 0.887 0.000 0.0 s > True 0.890 0.892 -0.002 0.2 s = 34.2 operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.369 0.327 0.042 -11.4 s == 'abc' 1.630 1.620 0.010 -0.6 s > 'abc' 1.640 1.620 0.020 -1.2 s is 4 0.372 0.332 0.040 -10.8 s == 4 0.791 0.795 -0.004 0.5 s > 4 0.797 0.798 -0.001 0.1 s is -1001 0.375 0.331 0.044 -11.7 s == -1001 0.792 0.792 0.000 0.0 s > -1001 0.790 0.791 -0.001 0.1 s is 34.7 0.367 0.482 -0.115 31.3 s == 34.7 1.080 0.536 0.544 -50.4 s > 34.7 0.560 0.621 -0.061 10.9 s is 'a b c' 0.387 0.337 0.050 -12.9 s == 'a b c' 1.760 1.710 0.050 -2.8 s > 'a b c' 1.710 1.680 0.030 -1.8 s is True 0.614 0.509 0.105 -17.1 s == True 1.050 1.020 0.030 -2.9 s > True 1.060 1.020 0.040 -3.8 s = 'a b c' operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.379 0.345 0.034 -9.0 s == 'abc' 0.542 0.494 0.048 -8.9 s > 'abc' 0.586 0.593 -0.007 1.2 s is 4 0.430 0.344 0.086 -20.0 s == 4 1.260 1.230 0.030 -2.4 s > 4 1.370 1.230 0.140 -10.2 s is -1001 0.431 0.372 0.059 -13.7 s == -1001 1.250 1.640 -0.390 31.2 s > -1001 1.240 1.260 -0.020 1.6 s is 34.7 0.383 0.337 0.046 -12.0 s == 34.7 1.770 1.680 0.090 -5.1 s > 34.7 1.670 1.660 0.010 -0.6 s is 'a b c' 0.423 0.376 0.047 -11.1 s == 'a b c' 0.506 0.510 -0.004 0.8 s > 'a b c' 0.517 0.564 -0.047 9.1 s is True 0.550 0.514 0.036 -6.5 s == True 1.470 1.640 -0.170 11.6 s > True 1.450 1.430 0.020 -1.4 s = object() operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.389 0.379 0.010 -2.6 s == 'abc' 1.220 1.370 -0.150 12.3 s > 'abc' 1.220 2.600 -1.380 113.1 s is 4 0.427 0.349 0.078 -18.3 s == 4 1.080 1.620 -0.540 50.0 s > 4 1.060 1.070 -0.010 0.9 s is -1001 0.437 0.343 0.094 -21.5 s == -1001 1.070 1.130 -0.060 5.6 s > -1001 1.060 1.090 -0.030 2.8 s is 34.7 0.419 0.338 0.081 -19.3 s == 34.7 1.710 1.520 0.190 -11.1 s > 34.7 1.520 1.540 -0.020 1.3 s is 'a b c' 0.380 0.347 0.033 -8.7 s == 'a b c' 2.020 1.210 0.810 -40.1 s > 'a b c' 1.260 1.210 0.050 -4.0 s is True 0.622 0.515 0.107 -17.2 s == True 1.220 1.220 0.000 0.0 s > True 1.210 1.210 0.000 0.0 s = [] operation before after delta %chg --------- ------ ----- ----- ---- s is 'abc' 0.369 0.326 0.043 -11.7 s == 'abc' 1.220 1.200 0.020 -1.6 s > 'abc' 1.220 1.200 0.020 -1.6 s is 4 0.372 0.332 0.040 -10.8 s == 4 1.160 1.150 0.010 -0.9 s > 4 1.150 1.150 0.000 0.0 s is -1001 0.371 0.334 0.037 -10.0 s == -1001 1.150 1.140 0.010 -0.9 s > -1001 1.150 1.150 0.000 0.0 s is 34.7 0.368 0.326 0.042 -11.4 s == 34.7 1.500 1.480 0.020 -1.3 s > 34.7 1.490 1.490 0.000 0.0 s is 'a b c' 0.366 0.325 0.041 -11.2 s == 'a b c' 1.220 1.200 0.020 -1.6 s > 'a b c' 1.220 1.200 0.020 -1.6 s is True 0.531 0.484 0.047 -8.9 s == True 1.360 1.350 0.010 -0.7 s > True 1.350 1.350 0.000 0.0 I fully expected that the "is" tests would be faster and without question the "==" and ">" tests would be slower. I was quite surprised that this wasn't always the case. The above tests were run on an 800MHz Powerbook G4 running Mac OSX 10.2.8. I don't have immediate access in Intel hardware, though I'll try to run these tests on cygwin this week. I'd be happy to be shown that my shell script isn't measuring what I think it's measuring as well. Skip ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-20 13:27 Message: Logged In: YES user_id=80475 Even "is" and "is not" are not helped by more than a couple of cycles. This fragment essentially inlines part of code for cmp_outcome(). Only the function call is saved. It does slow down other code paths by introducing an unpredictable branch. If the inlining were considered important, then the whole of cmp_outcome() should be inlined. Then, all comparisons save a single call/return pair. The cost is further increasing the size of the eval loop. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-20 12:45 Message: Logged In: YES user_id=31435 Well, there's little question that this will speed "is" and "is not", but it also slows all other cases by the cost of the switch-and-branch to determine that they're not the favored cases. So why should we believe that speeding "is" and "is not" is more important than slowing other cases? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=918462&group_id=5470 From noreply at sourceforge.net Tue Mar 23 02:30:55 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 02:31:00 2004 Subject: [Patches] [ python-Patches-921466 ] Reduce number of open calls on startup Message-ID: Patches item #921466, was opened at 2004-03-22 19:10 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921466&group_id=5470 Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Martin v. L?wis (loewis) Assigned to: Nobody/Anonymous (nobody) Summary: Reduce number of open calls on startup Initial Comment: This patch uses sys.path_importer_cache to reduce the number of open calls, in the following way: - if the value in path_importer_cache is None, it stats the path to find out whether the file exists - it then puts True/False into path_importer_cache - if the value in path_importer_cache is False, the path entry is skipped on all imports - if the value is True, the stat call is skipped, and open calls for files in the directory are made. On Linux, this reduces the number of open calls for an empty script from 343 to 263. The startup-time (for 100 interpreter invocations) goes down by one percent (from 0.0819s to 0.08113s per invocation). ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-23 02:30 Message: Logged In: YES user_id=80475 I am surprised that making 25% fewer open calls doesn't save more than 1% in startup time. One other thought, I wonder if the timing of these changes is affected by the OS keeping recently loaded files in buffers so that disk access time not included. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921466&group_id=5470 From noreply at sourceforge.net Tue Mar 23 10:05:51 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 10:06:02 2004 Subject: [Patches] [ python-Patches-921793 ] Rewrite of site.py Message-ID: Patches item #921793, was opened at 2004-03-23 07:05 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921793&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 3 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: Rewrite of site.py Initial Comment: Brought to you by the PyCon core sprint, I rewrote Lib/site.py to clean it up and modernize it. The biggest change is putting all the code into appropriate functions and defining a main() function that gets executed at import. Cleans up the code and allows others to run the functions after import in case they want the functionality. Also modernized it by using sets and universal newlines. A new test suite is also being included in this tracker item. You can partially run it against the original site.py, but some tests will fail since they call new functions. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921793&group_id=5470 From noreply at sourceforge.net Tue Mar 23 11:40:04 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 11:40:36 2004 Subject: [Patches] [ python-Patches-920211 ] Patch submission for #876533 (potential leak in ensure_froml Message-ID: Patches item #920211, was opened at 2004-03-20 21:41 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920211&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: Brian Leair (bleair2) Assigned to: Nobody/Anonymous (nobody) Summary: Patch submission for #876533 (potential leak in ensure_froml Initial Comment: Summary: (?) potential leak in ensure_fromlist (import.c) In the recursive call inside of ensure_fromlist () an early exit is taken if the recursive call returns 0. In this early exit case the object "all" needs to be Py_DECREF (). ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-23 17:40 Message: Logged In: YES user_id=21627 It appears that this patch is identical to the one included in the bugreport, which I have applied. So closing this as out-of-date ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920211&group_id=5470 From noreply at sourceforge.net Tue Mar 23 10:43:12 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 12:03:49 2004 Subject: [Patches] [ python-Patches-921466 ] Reduce number of open calls on startup Message-ID: Patches item #921466, was opened at 2004-03-23 01:10 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921466&group_id=5470 Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Martin v. L?wis (loewis) Assigned to: Nobody/Anonymous (nobody) Summary: Reduce number of open calls on startup Initial Comment: This patch uses sys.path_importer_cache to reduce the number of open calls, in the following way: - if the value in path_importer_cache is None, it stats the path to find out whether the file exists - it then puts True/False into path_importer_cache - if the value in path_importer_cache is False, the path entry is skipped on all imports - if the value is True, the stat call is skipped, and open calls for files in the directory are made. On Linux, this reduces the number of open calls for an empty script from 343 to 263. The startup-time (for 100 interpreter invocations) goes down by one percent (from 0.0819s to 0.08113s per invocation). ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-23 16:43 Message: Logged In: YES user_id=21627 It's certainly the case that the system has cached all files needed for startup in memory, including the directory contents of all directories searched. OTOH, I assume that is the scenario in which people worry about startup time: high-frequency invocations of python. For a single invocation, it shouldn't matter much whether it takes 0.04s or 0.08s. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-23 08:30 Message: Logged In: YES user_id=80475 I am surprised that making 25% fewer open calls doesn't save more than 1% in startup time. One other thought, I wonder if the timing of these changes is affected by the OS keeping recently loaded files in buffers so that disk access time not included. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921466&group_id=5470 From noreply at sourceforge.net Tue Mar 23 12:37:30 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 12:37:34 2004 Subject: [Patches] [ python-Patches-921793 ] Rewrite of site.py Message-ID: Patches item #921793, was opened at 2004-03-23 07:05 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921793&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 3 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: Rewrite of site.py Initial Comment: Brought to you by the PyCon core sprint, I rewrote Lib/site.py to clean it up and modernize it. The biggest change is putting all the code into appropriate functions and defining a main() function that gets executed at import. Cleans up the code and allows others to run the functions after import in case they want the functionality. Also modernized it by using sets and universal newlines. A new test suite is also being included in this tracker item. You can partially run it against the original site.py, but some tests will fail since they call new functions. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2004-03-23 09:37 Message: Logged In: YES user_id=357491 Just discovered one of the regression tests fails if you run all the tests with regrtest.py. Trying to fix now since it is a bug in the test. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921793&group_id=5470 From noreply at sourceforge.net Tue Mar 23 12:44:22 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 12:44:27 2004 Subject: [Patches] [ python-Patches-921793 ] Rewrite of site.py Message-ID: Patches item #921793, was opened at 2004-03-23 07:05 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921793&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 3 Submitted By: Brett Cannon (bcannon) Assigned to: Nobody/Anonymous (nobody) Summary: Rewrite of site.py Initial Comment: Brought to you by the PyCon core sprint, I rewrote Lib/site.py to clean it up and modernize it. The biggest change is putting all the code into appropriate functions and defining a main() function that gets executed at import. Cleans up the code and allows others to run the functions after import in case they want the functionality. Also modernized it by using sets and universal newlines. A new test suite is also being included in this tracker item. You can partially run it against the original site.py, but some tests will fail since they call new functions. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2004-03-23 09:44 Message: Logged In: YES user_id=357491 OK, now the test passes. Problem test was not matching what the code did by creating a normalized, absolute path of what was in sys.path . ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2004-03-23 09:37 Message: Logged In: YES user_id=357491 Just discovered one of the regression tests fails if you run all the tests with regrtest.py. Trying to fix now since it is a bug in the test. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921793&group_id=5470 From noreply at sourceforge.net Tue Mar 23 13:43:55 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 13:43:58 2004 Subject: [Patches] [ python-Patches-921927 ] Fixed a typo/thinko spelling "parameter" as "paramter" Message-ID: Patches item #921927, was opened at 2004-03-23 18:43 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921927&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sean Reifschneider (jafo) Assigned to: Nobody/Anonymous (nobody) Summary: Fixed a typo/thinko spelling "parameter" as "paramter" Initial Comment: Found by Bill Sconce, showing him how to submit a patch to SF. Mis-spelled "parameter" in doc.tex and concrete.tex. Sean ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921927&group_id=5470 From noreply at sourceforge.net Tue Mar 23 17:47:07 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 17:47:14 2004 Subject: [Patches] [ python-Patches-920509 ] Patch to enable profiling of C functions called from python Message-ID: Patches item #920509, was opened at 2004-03-21 10:25 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch to enable profiling of C functions called from python Initial Comment: This patch enables the profiling of C extension functions (and builtins) called via the CALL_FUNCTION opcode. This also includes a modification to lib/profile.py to handle the new call type. On my Powerbook 12" 867Mhz it incurs a 0.5% performance hit in the interpreter when profiling is not enabled. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-23 17:47 Message: Logged In: YES user_id=430343 New patch attached to this post (and old patch removed). This patch makes new events for all C-related calls, so old profilers will still function properly if they ignore event types they don't understand. Also fixed bdb, tested hotshot, and changed the documentation for the tracing API and the debugger event types to document the new event types (and fixed some errors in the names of the old events). Also provides new data for test_profile to compare against and modified test_profilehooks for minimal understanding of c calls so it still passes. regrtest passes all tests on MacOS X, and the non-profiling performance hit is around 0.5%. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-22 17:32 Message: Logged In: YES user_id=430343 I'm working on a few modifications to this patch for documentation, as well as fixing the messages resulting from the fact that bdb is being handed events it doesn't understand. I'll post that at some point before PyCon starts (tomorrow, probably). ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-21 10:28 Message: Logged In: YES user_id=430343 Note: this patch is fully backwards with profiler use - the data file is not changed in any way. Also, any 3rd party profilers which would handle unknown event types will still continue to work (the standard python profiler isn't one of those, so the patch includes a fix for that as well). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 From noreply at sourceforge.net Tue Mar 23 18:25:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 18:25:32 2004 Subject: [Patches] [ python-Patches-920509 ] Patch to enable profiling of C functions called from python Message-ID: Patches item #920509, was opened at 2004-03-21 10:25 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch to enable profiling of C functions called from python Initial Comment: This patch enables the profiling of C extension functions (and builtins) called via the CALL_FUNCTION opcode. This also includes a modification to lib/profile.py to handle the new call type. On my Powerbook 12" 867Mhz it incurs a 0.5% performance hit in the interpreter when profiling is not enabled. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-23 18:25 Message: Logged In: YES user_id=430343 BTW, I forgot to patch Misc/NEWS, so whoever commits this should touch it. Thanks. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-23 17:47 Message: Logged In: YES user_id=430343 New patch attached to this post (and old patch removed). This patch makes new events for all C-related calls, so old profilers will still function properly if they ignore event types they don't understand. Also fixed bdb, tested hotshot, and changed the documentation for the tracing API and the debugger event types to document the new event types (and fixed some errors in the names of the old events). Also provides new data for test_profile to compare against and modified test_profilehooks for minimal understanding of c calls so it still passes. regrtest passes all tests on MacOS X, and the non-profiling performance hit is around 0.5%. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-22 17:32 Message: Logged In: YES user_id=430343 I'm working on a few modifications to this patch for documentation, as well as fixing the messages resulting from the fact that bdb is being handed events it doesn't understand. I'll post that at some point before PyCon starts (tomorrow, probably). ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-21 10:28 Message: Logged In: YES user_id=430343 Note: this patch is fully backwards with profiler use - the data file is not changed in any way. Also, any 3rd party profilers which would handle unknown event types will still continue to work (the standard python profiler isn't one of those, so the patch includes a fix for that as well). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 From noreply at sourceforge.net Tue Mar 23 18:41:37 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 18:41:41 2004 Subject: [Patches] [ python-Patches-922115 ] PEP 292 reference implementation Message-ID: Patches item #922115, was opened at 2004-03-23 18:41 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922115&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Barry A. Warsaw (bwarsaw) Assigned to: Brett Cannon (bcannon) Summary: PEP 292 reference implementation Initial Comment: Here's a reference implementation for PEP 292, along with a few other things we talked about at PyCON II - creation of a top-level stringlib package - addition of stringlib/safedict.py which is a handy adjunct to PEP 292 I will also be checking in an updated PEP 292. Still to do: add test cases and documentation. I will do this in time for Python 2.4, if the PEP is accepted. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922115&group_id=5470 From noreply at sourceforge.net Tue Mar 23 18:44:19 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 18:44:22 2004 Subject: [Patches] [ python-Patches-922117 ] unidiomatic str.replace Message-ID: Patches item #922117, was opened at 2004-03-23 17:44 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Andrew Gaul (gaul) Assigned to: Nobody/Anonymous (nobody) Summary: unidiomatic str.replace Initial Comment: Some code in the standard library is using s = 'a'.join(s.split('b')) instead of s = s.replace('a', 'b'). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 From noreply at sourceforge.net Tue Mar 23 23:17:00 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 23 23:17:04 2004 Subject: [Patches] [ python-Patches-922167 ] Patch to 742342 Crash on recursive reload Message-ID: Patches item #922167, was opened at 2004-03-24 00:17 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922167&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Brian Leair (bleair2) Assigned to: Nobody/Anonymous (nobody) Summary: Patch to 742342 Crash on recursive reload Initial Comment: Modules that causes a recursive reload cause a program abort. Easy for python developer to fix in their code, but the python interpreter shouldn't crash. This patch guards against circular reloading. This protection comes by an additional dictionary stored in the interpreter state to keep trakc of modules that are reloading. Recursive imports are protected against in a similar manner ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922167&group_id=5470 From noreply at sourceforge.net Wed Mar 24 08:54:41 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 24 08:54:55 2004 Subject: [Patches] [ python-Patches-922117 ] unidiomatic str.replace Message-ID: Patches item #922117, was opened at 2004-03-24 00:44 Message generated for change (Settings changed) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Andrew Gaul (gaul) >Assigned to: Brett Cannon (bcannon) Summary: unidiomatic str.replace Initial Comment: Some code in the standard library is using s = 'a'.join(s.split('b')) instead of s = s.replace('a', 'b'). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 From noreply at sourceforge.net Wed Mar 24 08:56:12 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 24 08:56:16 2004 Subject: [Patches] [ python-Patches-922117 ] unidiomatic str.replace Message-ID: Patches item #922117, was opened at 2004-03-24 00:44 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Andrew Gaul (gaul) Assigned to: Brett Cannon (bcannon) Summary: unidiomatic str.replace Initial Comment: Some code in the standard library is using s = 'a'.join(s.split('b')) instead of s = s.replace('a', 'b'). ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-24 14:56 Message: Logged In: YES user_id=21627 Please submit unified or context diffs in the future. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 From noreply at sourceforge.net Wed Mar 24 09:12:12 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 24 09:12:21 2004 Subject: [Patches] [ python-Patches-920509 ] Patch to enable profiling of C functions called from python Message-ID: Patches item #920509, was opened at 2004-03-21 16:25 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open >Resolution: Accepted Priority: 5 Submitted By: Nick Bastin (mondragon) >Assigned to: Nick Bastin (mondragon) Summary: Patch to enable profiling of C functions called from python Initial Comment: This patch enables the profiling of C extension functions (and builtins) called via the CALL_FUNCTION opcode. This also includes a modification to lib/profile.py to handle the new call type. On my Powerbook 12" 867Mhz it incurs a 0.5% performance hit in the interpreter when profiling is not enabled. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-24 15:12 Message: Logged In: YES user_id=21627 The patch is fine, with the following changes: - use tabs instead of spaces for indentation in C source code - add the Misc/NEWS change ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-24 00:25 Message: Logged In: YES user_id=430343 BTW, I forgot to patch Misc/NEWS, so whoever commits this should touch it. Thanks. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-23 23:47 Message: Logged In: YES user_id=430343 New patch attached to this post (and old patch removed). This patch makes new events for all C-related calls, so old profilers will still function properly if they ignore event types they don't understand. Also fixed bdb, tested hotshot, and changed the documentation for the tracing API and the debugger event types to document the new event types (and fixed some errors in the names of the old events). Also provides new data for test_profile to compare against and modified test_profilehooks for minimal understanding of c calls so it still passes. regrtest passes all tests on MacOS X, and the non-profiling performance hit is around 0.5%. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-22 23:32 Message: Logged In: YES user_id=430343 I'm working on a few modifications to this patch for documentation, as well as fixing the messages resulting from the fact that bdb is being handed events it doesn't understand. I'll post that at some point before PyCon starts (tomorrow, probably). ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-21 16:28 Message: Logged In: YES user_id=430343 Note: this patch is fully backwards with profiler use - the data file is not changed in any way. Also, any 3rd party profilers which would handle unknown event types will still continue to work (the standard python profiler isn't one of those, so the patch includes a fix for that as well). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 From noreply at sourceforge.net Wed Mar 24 09:07:01 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 24 11:02:52 2004 Subject: [Patches] [ python-Patches-922117 ] unidiomatic str.replace Message-ID: Patches item #922117, was opened at 2004-03-23 15:44 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 >Category: Library (Lib) >Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Andrew Gaul (gaul) Assigned to: Brett Cannon (bcannon) Summary: unidiomatic str.replace Initial Comment: Some code in the standard library is using s = 'a'.join(s.split('b')) instead of s = s.replace('a', 'b'). ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2004-03-24 06:07 Message: Logged In: YES user_id=357491 Applied (don't have the rev. #s since SF went read-only when I first submitted this and then I lost the page I had with all of this filled out). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-03-24 05:56 Message: Logged In: YES user_id=21627 Please submit unified or context diffs in the future. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922117&group_id=5470 From prxjkvqdtn at hotmail.com Wed Mar 24 19:13:22 2004 From: prxjkvqdtn at hotmail.com (Derick Silva) Date: Wed Mar 24 16:12:45 2004 Subject: [Patches] finally have a big dick Message-ID: Drop the HAMMER on the next bitch you Nail.... http://biology.homesr45.com/vp5 Click here if you do not wish to be invited again: http://fightnottobesingle.com/remove/?oc=xxxxyyyy From noreply at sourceforge.net Wed Mar 24 17:10:01 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 24 17:13:43 2004 Subject: [Patches] [ python-Patches-920509 ] Patch to enable profiling of C functions called from python Message-ID: Patches item #920509, was opened at 2004-03-21 10:25 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nick Bastin (mondragon) Summary: Patch to enable profiling of C functions called from python Initial Comment: This patch enables the profiling of C extension functions (and builtins) called via the CALL_FUNCTION opcode. This also includes a modification to lib/profile.py to handle the new call type. On my Powerbook 12" 867Mhz it incurs a 0.5% performance hit in the interpreter when profiling is not enabled. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-24 17:10 Message: Logged In: YES user_id=430343 Files modified: init.tex 1.16 libpdb.tex 1.39 pystate.h 2.27 bdb.py 1.44 profile.py 1.55 test_profilehooks.py 1.11 output/test_profile 1.3 NEWS 1.961 ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-03-24 09:12 Message: Logged In: YES user_id=21627 The patch is fine, with the following changes: - use tabs instead of spaces for indentation in C source code - add the Misc/NEWS change ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-23 18:25 Message: Logged In: YES user_id=430343 BTW, I forgot to patch Misc/NEWS, so whoever commits this should touch it. Thanks. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-23 17:47 Message: Logged In: YES user_id=430343 New patch attached to this post (and old patch removed). This patch makes new events for all C-related calls, so old profilers will still function properly if they ignore event types they don't understand. Also fixed bdb, tested hotshot, and changed the documentation for the tracing API and the debugger event types to document the new event types (and fixed some errors in the names of the old events). Also provides new data for test_profile to compare against and modified test_profilehooks for minimal understanding of c calls so it still passes. regrtest passes all tests on MacOS X, and the non-profiling performance hit is around 0.5%. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-22 17:32 Message: Logged In: YES user_id=430343 I'm working on a few modifications to this patch for documentation, as well as fixing the messages resulting from the fact that bdb is being handed events it doesn't understand. I'll post that at some point before PyCon starts (tomorrow, probably). ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-21 10:28 Message: Logged In: YES user_id=430343 Note: this patch is fully backwards with profiler use - the data file is not changed in any way. Also, any 3rd party profilers which would handle unknown event types will still continue to work (the standard python profiler isn't one of those, so the patch includes a fix for that as well). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=920509&group_id=5470 From noreply at sourceforge.net Wed Mar 24 22:35:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 24 22:35:46 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From ventricle at ilovegeorgina.com Wed Mar 24 23:57:57 2004 From: ventricle at ilovegeorgina.com (Traffickers J. Runnel) Date: Wed Mar 24 23:59:38 2004 Subject: [Patches] Patches, Finest meedication for you! Message-ID: <001101c41225$50fe3e64$c7747f5b@ilovegeorgina.com> Guten Tag :)) Reagan won because he ran against Jimmy Carter. If he ran unopposed he would have lost. Patches, searching for a source to buy medicaticTon? Finest ViagTrHa and CialXtis Quick weight (mulls obtusifid) loss and anti depressant meedication! Best price on ValiumJm and Xanafox Best deals, 80 pernect off! We ship wdrlowide Here you will find it: http://www.hbgp.com/d13/index.php?id=d13 You are completely anonymous! And let the winds of the heavens dance between you. It is faith in something and enthusiasm for something that makes a life worth looking at. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040324/3e4196f0/attachment.html From noreply at sourceforge.net Thu Mar 25 04:10:10 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 04:11:27 2004 Subject: [Patches] [ python-Patches-921927 ] Fixed a typo/thinko spelling Message-ID: Patches item #921927, was opened at 2004-03-23 13:43 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921927&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Sean Reifschneider (jafo) Assigned to: Nobody/Anonymous (nobody) >Summary: Fixed a typo/thinko spelling Initial Comment: Found by Bill Sconce, showing him how to submit a patch to SF. Mis-spelled "parameter" in doc.tex and concrete.tex. Sean ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-25 04:10 Message: Logged In: YES user_id=80475 Fixed. Thanks for the report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921927&group_id=5470 From f3ennffnz at ya.com Thu Mar 25 03:39:23 2004 From: f3ennffnz at ya.com (Ali Heller) Date: Thu Mar 25 05:40:01 2004 Subject: [Patches] Re: Important schoolteacher Message-ID: <2t-1s$$v5893tw@70nobj0> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040325/002d9dff/attachment.html From noreply at sourceforge.net Thu Mar 25 09:37:19 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 10:02:41 2004 Subject: [Patches] [ python-Patches-919299 ] API Ref: PyErr_SetInterrupt not Obsolete Message-ID: Patches item #919299, was opened at 2004-03-19 01:21 Message generated for change (Comment added) made by fdrake You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919299&group_id=5470 Category: Documentation >Group: Python 2.3 Status: Open >Resolution: Fixed Priority: 5 Submitted By: Kurt B. Kaiser (kbk) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: API Ref: PyErr_SetInterrupt not Obsolete Initial Comment: The API document Section 4. claims that this function is obsolete, but it is used in thread.interrupt_main(), which in turn is used in IDLE. Attached patch removes the 'obsolete' statement. If there is some reason that it is going to disappear, then we have to figure something out for thread.interrupt_main. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2004-03-25 09:37 Message: Logged In: YES user_id=3066 Fixed in Doc/api/exceptions.tex 1.17. Someone should backport the change to Python 2.3.x and close this bug report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=919299&group_id=5470 From noreply at sourceforge.net Thu Mar 25 10:10:40 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 10:10:44 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 10:13:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 10:13:29 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 11:36:47 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 11:36:51 2004 Subject: [Patches] [ python-Patches-923236 ] 'os' patch to speed up import (and python startup) Message-ID: Patches item #923236, was opened at 2004-03-25 11:36 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923236&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: 'os' patch to speed up import (and python startup) Initial Comment: This patch changes the _Environ subclass of UserDict to use dict instead, which avoids the import of UserDict, save 6-8ms on python startup with site.py (~4% improvement). I tested this on my MacOS X box, but since this code is really only used in Win32, someone should test it there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923236&group_id=5470 From noreply at sourceforge.net Thu Mar 25 11:37:47 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 11:37:53 2004 Subject: [Patches] [ python-Patches-923236 ] 'os' patch to speed up import (and python startup) Message-ID: Patches item #923236, was opened at 2004-03-25 11:36 Message generated for change (Settings changed) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923236&group_id=5470 Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: 'os' patch to speed up import (and python startup) Initial Comment: This patch changes the _Environ subclass of UserDict to use dict instead, which avoids the import of UserDict, save 6-8ms on python startup with site.py (~4% improvement). I tested this on my MacOS X box, but since this code is really only used in Win32, someone should test it there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923236&group_id=5470 From noreply at sourceforge.net Thu Mar 25 11:54:57 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 11:55:03 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-25 11:54 Message: Logged In: YES user_id=430343 I've attached a new patch which corrects the multiple import lock checks, and uses a global binding instead of the static mutable default arg hack.. :-) ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 10:34 Message: Logged In: YES user_id=430343 Yes, Jim's patch fixes the obvious problem...that's what I get for writing code after a long day at PyCon.. :-) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 12:00:02 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 12:00:18 2004 Subject: [Patches] [ python-Patches-873224 ] Marshal clean-up Message-ID: Patches item #873224, was opened at 2004-01-08 19:29 Message generated for change (Settings changed) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873224&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 2 Submitted By: Armin Rigo (arigo) >Assigned to: Martin v. L?wis (loewis) Summary: Marshal clean-up Initial Comment: Reading random bytes with the marshal module can segfault the interpreter. Moreover, reading a truncated .pyc file can also successfully return a (corrupted) object instead of raising an error. Please review the attached patch if you consider this to be worth fixing. The patch also fixes a minor bug in import.c, which would sometimes write incomplete .pyc files in case of write error (and hence the truncated .pyc file problems could actually show up, in theory). ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-08 20:00 Message: Logged In: YES user_id=4771 Test cases: >>> marshal.loads('0') -> SystemError >>> marshal.loads('f') -> segfault >>> marshal.dumps(5L) 'l\x01\x00\x00\x00\x05\x00' >>> marshal.loads(_[:-1]) 65285L ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-08 19:55 Message: Logged In: YES user_id=6656 This patch results from my comments on IRC, so I'll take a look at it. Needs testcases! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873224&group_id=5470 From noreply at sourceforge.net Thu Mar 25 11:18:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 12:03:33 2004 Subject: [Patches] [ python-Patches-923226 ] 'os' patch to speed up import (and python startup) Message-ID: Patches item #923226, was opened at 2004-03-25 11:18 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923226&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: 'os' patch to speed up import (and python startup) Initial Comment: This patch changes the _Environ subclass of UserDict to use dict instead, which avoids the import of UserDict, save 6-8ms on python startup with site.py (~4% improvement). I tested this on my MacOS X box, but since this code is really only used in Win32, someone should test it there. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923226&group_id=5470 From noreply at sourceforge.net Thu Mar 25 08:25:09 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 12:03:49 2004 Subject: [Patches] [ python-Patches-923098 ] Support for interned strings in marshal Message-ID: Patches item #923098, was opened at 2004-03-25 14:25 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923098&group_id=5470 Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Martin v. L?wis (loewis) Assigned to: Nobody/Anonymous (nobody) Summary: Support for interned strings in marshal Initial Comment: This patch performs string sharing in marshal for interned strings. On marshalling, TYPE_INTERNED is generated for the first occurrence of an interned strings, and TYPE_STRINGREF for a later occurrence. On unmarshalling, TYPE_INTERNED strings are interned on unmarshalling. During marshalling, a dictionary is created to track the strings; on unmarshalling, a list. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923098&group_id=5470 From noreply at sourceforge.net Thu Mar 25 10:56:47 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 12:03:58 2004 Subject: [Patches] [ python-Patches-798638 ] Improve "veryhigh.tex" API docs Message-ID: Patches item #798638, was opened at 2003-09-01 12:11 Message generated for change (Comment added) made by fdrake You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=798638&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Jeff Epler (jepler) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Improve "veryhigh.tex" API docs Initial Comment: Many variant versions of functions were not documented in this chapter, and the PyCompilerFlags structure was not documented at all. This patch fixes that. It adds documentation for: +\begin{cfuncdesc}{int}{PyRun_AnyFileFlags}{FILE *fp, const char *filename, +\begin{cfuncdesc}{int}{PyRun_AnyFileEx}{FILE *fp, const char *filename, +\begin{cfuncdesc}{int}{PyRun_AnyFileExFlags}{FILE *fp, const char *filename, +\begin{cfuncdesc}{int}{PyRun_SimpleStringFlags}{const char *command, +\begin{cfuncdesc}{int}{PyRun_SimpleFileFlags}{FILE *fp, const char *filename, +\begin{cfuncdesc}{int}{PyRun_SimpleFileEx}{FILE *fp, const char *filename, +\begin{cfuncdesc}{int}{PyRun_SimpleFileExFlags}{FILE *fp, const char *filename, +\begin{cfuncdesc}{int}{PyRun_InteractiveOneFlags}{FILE *fp, +\begin{cfuncdesc}{int}{PyRun_InteractiveLoopFlags}{FILE *fp, +\begin{cfuncdesc}{struct _node*}{PyParser_SimpleParseStringFlags}{ +\begin{cfuncdesc}{struct _node*}{PyParser_SimpleParseStringFlagsFilename}{ +\begin{cfuncdesc}{struct _node*}{PyParser_SimpleParseFileFlags}{FILE *fp, +\begin{cfuncdesc}{PyObject*}{PyRun_StringFlags}{const char *str, int start, +\begin{cfuncdesc}{PyObject*}{PyRun_FileEx}{FILE *fp, const char *filename, +\begin{cfuncdesc}{PyObject*}{PyRun_FileFlags}{FILE *fp, const char *filename, +\begin{cfuncdesc}{PyObject*}{PyRun_FileExFlags}{FILE *fp, const char *filename, +\begin{cfuncdesc}{PyObject*}{Py_CompileStringFlags}{char *str, +\begin{ctypedesc}[PyCompilerFlags]{struct PyCompilerFlags} +\begin{cvardesc}{int}{CO_FUTURE_DIVISION} This patch also reflects the "const"ness of arguments to these functions. latex seemed to process the modified documentation just fine. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2004-03-25 10:56 Message: Logged In: YES user_id=3066 Committed a slightly modified version of the patch as Doc/api/veryhigh.tex 1.2. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=798638&group_id=5470 From noreply at sourceforge.net Thu Mar 25 11:15:26 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 12:04:06 2004 Subject: [Patches] [ python-Patches-732174 ] build of html docs broken (liboptparse.tex) Message-ID: Patches item #732174, was opened at 2003-05-04 09:47 Message generated for change (Comment added) made by fdrake You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=732174&group_id=5470 Category: Documentation Group: Python 2.3 >Status: Closed Resolution: Fixed Priority: 5 Submitted By: Matthias Klose (doko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: build of html docs broken (liboptparse.tex) Initial Comment: The last known working versions is 1.6. 1.7 doesn't build anymore. latex2html returns [...] Reading aux file: /build/packages/python2.3/python2.3-2.2.102/Doc/lib.aux ... Processing macros ...,,,,++................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Subroutine do_cmd_leftmargin redefined at (eval 732) line 1. defining handler for \leftmargin using \leftmargini .............Undefined subroutine &main::do_cmd_leftmargini called at (eval 732) line 1. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2004-03-25 11:15 Message: Logged In: YES user_id=3066 I'm not sure why the comments on this report discuss the info documentation when the original report is clearly about the HTML documentation. As far as I can tell, the HTML is currently being generated without errors. If I've missed something, please open a new report with pertinent information. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2003-07-16 00:22 Message: Logged In: YES user_id=3066 Current status: - "Python Library Reference" doesn't format, and I don't think this is the libre.tex problem. I get the message "Args out of range: 406147, 406149". I don't know what this refers to. - "Documenting Python" doesn't convert because of a couple of highly specialized macros that are defined locally within other macros with really short names. I don't really want to support them directly since they have such short names ("p" and "op"), though I did cave for the HTML conversion. - "Installing Python Modules" doesn't convert because it uses a specialized macro defined inline in the document ("installscheme"), but I don't really want to replace that in the document with the standard table markup. - "What's New in Python 2.X" is now built; any verision back to whatsnew21.tex can be formatted; the 2.0.x version isn't worth any work to make it format at this point. Only the current version is formatted by default; alternate versions can be specified using the WHATSNEW make variable. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2003-07-02 10:46 Message: Logged In: YES user_id=3066 Several parts of the patch are now committed, but not everything. In particular, the changes to Doc/lib/libre.tex have not been committed, pending time for me to take a closer look. ---------------------------------------------------------------------- Comment By: Matthias Klose (doko) Date: 2003-05-18 16:46 Message: Logged In: YES user_id=60903 Which version? py2texi.el is at 1.3 and unchanged the last six months. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-18 16:28 Message: Logged In: YES user_id=80475 The latest version is 1.9. Fred made changes to make it build again. ---------------------------------------------------------------------- Comment By: Matthias Klose (doko) Date: 2003-05-18 15:08 Message: Logged In: YES user_id=60903 Attached is a patch to fix the build of the info docs. The change to libre.tex maybe controversial, as it works around a problem in py2texi.el. ---------------------------------------------------------------------- Comment By: Matthias Klose (doko) Date: 2003-05-18 15:02 Message: Logged In: YES user_id=60903 Reopened. Please practice on other reports ;-) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2003-05-17 21:51 Message: Logged In: YES user_id=31435 Assigning to Raymond for practice in finding the Submit button . ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=732174&group_id=5470 From noreply at sourceforge.net Thu Mar 25 11:07:14 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 12:04:10 2004 Subject: [Patches] [ python-Patches-921318 ] Patch for Vinay Sajip's Python-logging package Message-ID: Patches item #921318, was opened at 2004-03-22 14:58 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921318&group_id=5470 Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Mark E. Davidson (medavidson) Assigned to: Vinay Sajip (vsajip) Summary: Patch for Vinay Sajip's Python-logging package Initial Comment: This is a patch for Vinay Sajip's Python-logging package. It adds the support for time-based log rollovers. Specifically, it adds a new handler called TimedRotatingFIleHandler. TimedRotatingFileHandler works just like the existing RotatingFileHandler, except that instead of rolling over when the log file hits a certain size, it rolls over when a time-interval has occurred. You can have the log files rollover ever seconds, minutes, hours, days. You can have log files rollover at midnight. Log files can also roll over on a day of the week. The name of the log file is customized to fit the interval specified. In other words, it's suffix is changed to match the interval you asked for. If you asked for it to roll over every hour, then the log file has a suffix with the year, month, day and hour, but if you roll over on a certain minute, then the suffix also has the minute. I had an extended correspondence with Vinay about this code as I was writing it, so he knows it's coming. It's been mostly tested, but not every variation has been exercised. The rollovers at seconds, minutes, hours have been exercised, as well as the midnight rollover, but the not the day of week. Additionally, the backupCount variation of the TimedRotatingFileHandler is somewhat simplistic and not well tested. This patch only affects handlers.py. It does NOT patch the graphical TkInter-based log configuration utility, as I ran out of time to work on this and Vinay said he might change the class layout. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 11:07 Message: Logged In: YES user_id=764593 (1) I think the H in the suffixes need to be capitalized. >>> time.strftime("%Y-%m-%d_%h-%M-%S", tt) '2004-03-25_-53-07' >>> time.strftime("%Y-%m-%d_%H-%M-%S", tt) '2004-03-25_10-53-07' (2) When globbing for old logfiles to delete, please make it more specfic. For instance, all suffixes (for the next century) start with "20", so you could use s = glob.glob(self.basFilename + ".20*") This will make it a bit less dangerous for someone to copy a file as basename.save when they want to save the info for debugging later. (3) The midnight and weekly versions should recalculate the rolloverAt instead of adding interval. This will keep them from getting out of synch if there is a maintenance window or daylight savings time change. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921318&group_id=5470 From noreply at sourceforge.net Thu Mar 25 12:35:54 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 12:36:03 2004 Subject: [Patches] [ python-Patches-876130 ] add C API to datetime module Message-ID: Patches item #876130, was opened at 2004-01-13 16:20 Message generated for change (Comment added) made by lemburg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876130&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Tuininga (atuining) Assigned to: M.-A. Lemburg (lemburg) Summary: add C API to datetime module Initial Comment: The datetime module does not have a C API which means that modules written in C cannot take advantage of the datetime module without incurring significant overhead. These patches supply this lack and are based on the C API exported by the mxDateTime module and the cStringIO module and enhanced as suggested by Guido. ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2004-03-25 18:35 Message: Logged In: YES user_id=38388 Anthony, have you had a chance to look at the comments ? ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-02-26 16:06 Message: Logged In: YES user_id=619560 Oops! It looks like my own misunderstanding of SourceForge and how it works is to blame. :-( I'll take a look at this and get back to you as soon as possible -- ignore my previous comment. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-02-26 16:03 Message: Logged In: YES user_id=619560 You haven't yet responded to my previous comments -- it looks like SourceForge put them __after__ your comments so it is quite understandable why you didn't notice them. If you can give me the answers to those questions or provide additional questions that I need to answer, then I think we can make progress again. I was waiting for __you__ :-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-02-26 00:08 Message: Logged In: YES user_id=38388 Any progress on this ? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-01-28 12:49 Message: Logged In: YES user_id=38388 Thanks for the clarification. I had forgotten about the macros -- I'd suggest that you document them in the datetimeapi.h include file to not cause the same confusion on the developer side (the trick to achieve adoption is to make things easy on the developer). As for TimeFromTicks(): This should be rather straight forward to implement: you take the time part of the ticks timestamp and create a time object from it. This is the way the DB API constructor works. Could you add C APIs for the three DB API ticks interface methods ? Other things that are missing: * documentation of the way the CAPI object can be used in datetimeapi.h * documentation of the various CAPI entries * documentation of the API magic number (there should be some scheme for this) I think that datetimeapi.h should be enough for a developer to read in order to use the interface. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-01-26 22:02 Message: Logged In: YES user_id=619560 I didn't think any additional API would be necessary for extracting date time information since the following macros are available: PyDateTime_GET_YEAR(o) PyDateTime_GET_MONTH(o) PyDateTime_GET_DAY(o) PyDateTime_DATE_GET_HOUR(o) PyDateTime_DATE_GET_MINUTE(o) PyDateTime_DATE_GET_SECOND(o) PyDateTime_DATE_GET_MICROSECOND(o) PyDateTime_TIME_GET_HOUR(o) PyDateTime_TIME_GET_MINUTE(o) PyDateTime_TIME_GET_SECOND(o) PyDateTime_TIME_GET_MICROSECOND(o) Were you thinking of something else that is needed? As for interfacing at the C level for converting from Unix ticks, the following method is already available and could simply be used as a drop in replacement for DateFromTicks() and TimestampFromTicks() from the DB API document. DateFromTicks() --> datetime.date.fromtimestamp() TimestampFromTicks() --> datetime.datetime.fromtimestamp() TimeFromTicks() is not already exposed but discussions with Tim Peters already suggested that would not happen since the concept is rather strange. You'll have to discuss that with him if you disagree! :-) Any comments? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-01-26 21:46 Message: Logged In: YES user_id=38388 This is a good start. However, you should add more APIs for extracting date/time information to the C API. mxDateTime.h from egenix-mx-base can provide a good basis for this. If you want to make the datetime module usable for database interfacing at C level, you'll also have to add interfaces for Unix ticks conversion, since these are often used by database C level interfaces. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-01-26 19:36 Message: Logged In: YES user_id=619560 I have no objection to providing patches for doc and test. A quick look at _testcapimodule.c doesn't provide any obvious ideas as to how to patch it. I looked for cStringIO which has a C API already and found nothing in there at all. The documentation also simply says "see the source for the information you require". Any suggestions as to how to proceed? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-01-26 04:17 Message: Logged In: YES user_id=31435 Marc-Andre, can you review this? I think you have more experience building C APIs for modules than anyone else (while I don't have any). There's certainly no objection to giving datetime a C API, and Guido already said he doesn't want to rely on that Martin changed datetime to be built as part of the core (in 2.3 on Windows, datetime.pyd was a distinct DLL; in CVS, datetime is compiled into the core DLL now). Anthony, you probably need to add doc and test patches. _testcapimodule.c holds tests of things you can only get at from C. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876130&group_id=5470 From noreply at sourceforge.net Thu Mar 25 10:34:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 13:03:17 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-25 10:34 Message: Logged In: YES user_id=430343 Yes, Jim's patch fixes the obvious problem...that's what I get for writing code after a long day at PyCon.. :-) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 13:07:02 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 13:07:11 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-25 13:07 Message: Logged In: YES user_id=430343 Apparently the file didn't stick last time. Lets try it again. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 11:54 Message: Logged In: YES user_id=430343 I've attached a new patch which corrects the multiple import lock checks, and uses a global binding instead of the static mutable default arg hack.. :-) ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 10:34 Message: Logged In: YES user_id=430343 Yes, Jim's patch fixes the obvious problem...that's what I get for writing code after a long day at PyCon.. :-) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 13:08:12 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 13:08:19 2004 Subject: [Patches] [ python-Patches-876130 ] add C API to datetime module Message-ID: Patches item #876130, was opened at 2004-01-13 08:20 Message generated for change (Comment added) made by atuining You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876130&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Tuininga (atuining) Assigned to: M.-A. Lemburg (lemburg) Summary: add C API to datetime module Initial Comment: The datetime module does not have a C API which means that modules written in C cannot take advantage of the datetime module without incurring significant overhead. These patches supply this lack and are based on the C API exported by the mxDateTime module and the cStringIO module and enhanced as suggested by Guido. ---------------------------------------------------------------------- >Comment By: Anthony Tuininga (atuining) Date: 2004-03-25 11:08 Message: Logged In: YES user_id=619560 Sorry for the delay. Things here have been quite busy recently both at work and at home. Let me see if I can answer your questions. 1) are you suggesting comments that include the macro names in datetimeapi.h similar to what I put in the comments below? Or something else? See modified datetimeapi.h for more info. 2) TimeFromTicks() is a very DB API specific routine. In chatting with others about the datetime module, they were not keen on adding such a beast. I guess there will have to be some discussion about whether datetimeapi.h is also a sort of half implementation for a C module implementing the DB API. I was intending to do this in my cx_Oracle module but not in datetimeapi.h. Could you clarify? 3) I could add C APIS for the three DB API ticks interfaces but I was simply intending to do that in cx_Oracle. Again, see #2. I can do it either way. :-) 4) I have added limited documentation in datetimeapi.h but is that really the right place for it? Is it even remotely close to what is needed? I'm not sure what is needed here exactly. 5) the API magic number is simply an arbitrary number which is stored in datetime.h and need not be understood by the implementor. For your information I simply took the name "datetime", converted each letter to its ordinal value in the alphabet, modded by 16 and used that hex character -- does that make any sense? This can be changed to whatever makes sense, though. 6) Whether or not datetimeapi.h should be sufficient for a developer or whether he should look at documentation in the usual locations (html pages, for example) I'll leave up to the Python development team. Let me know how I can assist. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-03-25 10:35 Message: Logged In: YES user_id=38388 Anthony, have you had a chance to look at the comments ? ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-02-26 08:06 Message: Logged In: YES user_id=619560 Oops! It looks like my own misunderstanding of SourceForge and how it works is to blame. :-( I'll take a look at this and get back to you as soon as possible -- ignore my previous comment. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-02-26 08:03 Message: Logged In: YES user_id=619560 You haven't yet responded to my previous comments -- it looks like SourceForge put them __after__ your comments so it is quite understandable why you didn't notice them. If you can give me the answers to those questions or provide additional questions that I need to answer, then I think we can make progress again. I was waiting for __you__ :-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-02-25 16:08 Message: Logged In: YES user_id=38388 Any progress on this ? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-01-28 04:49 Message: Logged In: YES user_id=38388 Thanks for the clarification. I had forgotten about the macros -- I'd suggest that you document them in the datetimeapi.h include file to not cause the same confusion on the developer side (the trick to achieve adoption is to make things easy on the developer). As for TimeFromTicks(): This should be rather straight forward to implement: you take the time part of the ticks timestamp and create a time object from it. This is the way the DB API constructor works. Could you add C APIs for the three DB API ticks interface methods ? Other things that are missing: * documentation of the way the CAPI object can be used in datetimeapi.h * documentation of the various CAPI entries * documentation of the API magic number (there should be some scheme for this) I think that datetimeapi.h should be enough for a developer to read in order to use the interface. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-01-26 14:02 Message: Logged In: YES user_id=619560 I didn't think any additional API would be necessary for extracting date time information since the following macros are available: PyDateTime_GET_YEAR(o) PyDateTime_GET_MONTH(o) PyDateTime_GET_DAY(o) PyDateTime_DATE_GET_HOUR(o) PyDateTime_DATE_GET_MINUTE(o) PyDateTime_DATE_GET_SECOND(o) PyDateTime_DATE_GET_MICROSECOND(o) PyDateTime_TIME_GET_HOUR(o) PyDateTime_TIME_GET_MINUTE(o) PyDateTime_TIME_GET_SECOND(o) PyDateTime_TIME_GET_MICROSECOND(o) Were you thinking of something else that is needed? As for interfacing at the C level for converting from Unix ticks, the following method is already available and could simply be used as a drop in replacement for DateFromTicks() and TimestampFromTicks() from the DB API document. DateFromTicks() --> datetime.date.fromtimestamp() TimestampFromTicks() --> datetime.datetime.fromtimestamp() TimeFromTicks() is not already exposed but discussions with Tim Peters already suggested that would not happen since the concept is rather strange. You'll have to discuss that with him if you disagree! :-) Any comments? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-01-26 13:46 Message: Logged In: YES user_id=38388 This is a good start. However, you should add more APIs for extracting date/time information to the C API. mxDateTime.h from egenix-mx-base can provide a good basis for this. If you want to make the datetime module usable for database interfacing at C level, you'll also have to add interfaces for Unix ticks conversion, since these are often used by database C level interfaces. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-01-26 11:36 Message: Logged In: YES user_id=619560 I have no objection to providing patches for doc and test. A quick look at _testcapimodule.c doesn't provide any obvious ideas as to how to patch it. I looked for cStringIO which has a C API already and found nothing in there at all. The documentation also simply says "see the source for the information you require". Any suggestions as to how to proceed? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-01-25 20:17 Message: Logged In: YES user_id=31435 Marc-Andre, can you review this? I think you have more experience building C APIs for modules than anyone else (while I don't have any). There's certainly no objection to giving datetime a C API, and Guido already said he doesn't want to rely on that Martin changed datetime to be built as part of the core (in 2.3 on Windows, datetime.pyd was a distinct DLL; in CVS, datetime is compiled into the core DLL now). Anthony, you probably need to add doc and test patches. _testcapimodule.c holds tests of things you can only get at from C. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876130&group_id=5470 From noreply at sourceforge.net Thu Mar 25 13:23:06 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 13:23:18 2004 Subject: [Patches] [ python-Patches-876130 ] add C API to datetime module Message-ID: Patches item #876130, was opened at 2004-01-13 16:20 Message generated for change (Comment added) made by lemburg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876130&group_id=5470 Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Tuininga (atuining) Assigned to: M.-A. Lemburg (lemburg) Summary: add C API to datetime module Initial Comment: The datetime module does not have a C API which means that modules written in C cannot take advantage of the datetime module without incurring significant overhead. These patches supply this lack and are based on the C API exported by the mxDateTime module and the cStringIO module and enhanced as suggested by Guido. ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2004-03-25 19:23 Message: Logged In: YES user_id=38388 Thanks for the comments... 1) Please put *all* the code for the C API into datetimeapi.h. I don't like your current mix of putting some things into datetime.h and others into datetimeapi.h (e.g. the PyDateTime_CAPI definition). 2) Since it's the only API missing if you want to use datetime objects in database interfacing, please add it. 3) dito. 4) Yes, the header file is the right place. Have a look at unicodeobject.h for what I have in mind (or mxDateTime.h for that matter). 5) Why add yet another magic number ? Isn't the Python API number enough ? 6) Since you don't provide a doc patch, please put comments into the header file. There can never be enough documentation and developers tend to look at the code rather than the docs ;-) ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-03-25 19:08 Message: Logged In: YES user_id=619560 Sorry for the delay. Things here have been quite busy recently both at work and at home. Let me see if I can answer your questions. 1) are you suggesting comments that include the macro names in datetimeapi.h similar to what I put in the comments below? Or something else? See modified datetimeapi.h for more info. 2) TimeFromTicks() is a very DB API specific routine. In chatting with others about the datetime module, they were not keen on adding such a beast. I guess there will have to be some discussion about whether datetimeapi.h is also a sort of half implementation for a C module implementing the DB API. I was intending to do this in my cx_Oracle module but not in datetimeapi.h. Could you clarify? 3) I could add C APIS for the three DB API ticks interfaces but I was simply intending to do that in cx_Oracle. Again, see #2. I can do it either way. :-) 4) I have added limited documentation in datetimeapi.h but is that really the right place for it? Is it even remotely close to what is needed? I'm not sure what is needed here exactly. 5) the API magic number is simply an arbitrary number which is stored in datetime.h and need not be understood by the implementor. For your information I simply took the name "datetime", converted each letter to its ordinal value in the alphabet, modded by 16 and used that hex character -- does that make any sense? This can be changed to whatever makes sense, though. 6) Whether or not datetimeapi.h should be sufficient for a developer or whether he should look at documentation in the usual locations (html pages, for example) I'll leave up to the Python development team. Let me know how I can assist. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-03-25 18:35 Message: Logged In: YES user_id=38388 Anthony, have you had a chance to look at the comments ? ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-02-26 16:06 Message: Logged In: YES user_id=619560 Oops! It looks like my own misunderstanding of SourceForge and how it works is to blame. :-( I'll take a look at this and get back to you as soon as possible -- ignore my previous comment. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-02-26 16:03 Message: Logged In: YES user_id=619560 You haven't yet responded to my previous comments -- it looks like SourceForge put them __after__ your comments so it is quite understandable why you didn't notice them. If you can give me the answers to those questions or provide additional questions that I need to answer, then I think we can make progress again. I was waiting for __you__ :-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-02-26 00:08 Message: Logged In: YES user_id=38388 Any progress on this ? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-01-28 12:49 Message: Logged In: YES user_id=38388 Thanks for the clarification. I had forgotten about the macros -- I'd suggest that you document them in the datetimeapi.h include file to not cause the same confusion on the developer side (the trick to achieve adoption is to make things easy on the developer). As for TimeFromTicks(): This should be rather straight forward to implement: you take the time part of the ticks timestamp and create a time object from it. This is the way the DB API constructor works. Could you add C APIs for the three DB API ticks interface methods ? Other things that are missing: * documentation of the way the CAPI object can be used in datetimeapi.h * documentation of the various CAPI entries * documentation of the API magic number (there should be some scheme for this) I think that datetimeapi.h should be enough for a developer to read in order to use the interface. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-01-26 22:02 Message: Logged In: YES user_id=619560 I didn't think any additional API would be necessary for extracting date time information since the following macros are available: PyDateTime_GET_YEAR(o) PyDateTime_GET_MONTH(o) PyDateTime_GET_DAY(o) PyDateTime_DATE_GET_HOUR(o) PyDateTime_DATE_GET_MINUTE(o) PyDateTime_DATE_GET_SECOND(o) PyDateTime_DATE_GET_MICROSECOND(o) PyDateTime_TIME_GET_HOUR(o) PyDateTime_TIME_GET_MINUTE(o) PyDateTime_TIME_GET_SECOND(o) PyDateTime_TIME_GET_MICROSECOND(o) Were you thinking of something else that is needed? As for interfacing at the C level for converting from Unix ticks, the following method is already available and could simply be used as a drop in replacement for DateFromTicks() and TimestampFromTicks() from the DB API document. DateFromTicks() --> datetime.date.fromtimestamp() TimestampFromTicks() --> datetime.datetime.fromtimestamp() TimeFromTicks() is not already exposed but discussions with Tim Peters already suggested that would not happen since the concept is rather strange. You'll have to discuss that with him if you disagree! :-) Any comments? ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2004-01-26 21:46 Message: Logged In: YES user_id=38388 This is a good start. However, you should add more APIs for extracting date/time information to the C API. mxDateTime.h from egenix-mx-base can provide a good basis for this. If you want to make the datetime module usable for database interfacing at C level, you'll also have to add interfaces for Unix ticks conversion, since these are often used by database C level interfaces. ---------------------------------------------------------------------- Comment By: Anthony Tuininga (atuining) Date: 2004-01-26 19:36 Message: Logged In: YES user_id=619560 I have no objection to providing patches for doc and test. A quick look at _testcapimodule.c doesn't provide any obvious ideas as to how to patch it. I looked for cStringIO which has a C API already and found nothing in there at all. The documentation also simply says "see the source for the information you require". Any suggestions as to how to proceed? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-01-26 04:17 Message: Logged In: YES user_id=31435 Marc-Andre, can you review this? I think you have more experience building C APIs for modules than anyone else (while I don't have any). There's certainly no objection to giving datetime a C API, and Guido already said he doesn't want to rely on that Martin changed datetime to be built as part of the core (in 2.3 on Windows, datetime.pyd was a distinct DLL; in CVS, datetime is compiled into the core DLL now). Anthony, you probably need to add doc and test patches. _testcapimodule.c holds tests of things you can only get at from C. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=876130&group_id=5470 From noreply at sourceforge.net Thu Mar 25 14:14:36 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 14:15:00 2004 Subject: [Patches] [ python-Patches-921318 ] Patch for Vinay Sajip's Python-logging package Message-ID: Patches item #921318, was opened at 2004-03-22 11:58 Message generated for change (Comment added) made by medavidson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921318&group_id=5470 Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Mark E. Davidson (medavidson) Assigned to: Vinay Sajip (vsajip) Summary: Patch for Vinay Sajip's Python-logging package Initial Comment: This is a patch for Vinay Sajip's Python-logging package. It adds the support for time-based log rollovers. Specifically, it adds a new handler called TimedRotatingFIleHandler. TimedRotatingFileHandler works just like the existing RotatingFileHandler, except that instead of rolling over when the log file hits a certain size, it rolls over when a time-interval has occurred. You can have the log files rollover ever seconds, minutes, hours, days. You can have log files rollover at midnight. Log files can also roll over on a day of the week. The name of the log file is customized to fit the interval specified. In other words, it's suffix is changed to match the interval you asked for. If you asked for it to roll over every hour, then the log file has a suffix with the year, month, day and hour, but if you roll over on a certain minute, then the suffix also has the minute. I had an extended correspondence with Vinay about this code as I was writing it, so he knows it's coming. It's been mostly tested, but not every variation has been exercised. The rollovers at seconds, minutes, hours have been exercised, as well as the midnight rollover, but the not the day of week. Additionally, the backupCount variation of the TimedRotatingFileHandler is somewhat simplistic and not well tested. This patch only affects handlers.py. It does NOT patch the graphical TkInter-based log configuration utility, as I ran out of time to work on this and Vinay said he might change the class layout. ---------------------------------------------------------------------- >Comment By: Mark E. Davidson (medavidson) Date: 2004-03-25 11:14 Message: Logged In: YES user_id=877141 jimjjewitt is correct. I will try to fix these three issues and upload a new patch in the next day or so. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 08:07 Message: Logged In: YES user_id=764593 (1) I think the H in the suffixes need to be capitalized. >>> time.strftime("%Y-%m-%d_%h-%M-%S", tt) '2004-03-25_-53-07' >>> time.strftime("%Y-%m-%d_%H-%M-%S", tt) '2004-03-25_10-53-07' (2) When globbing for old logfiles to delete, please make it more specfic. For instance, all suffixes (for the next century) start with "20", so you could use s = glob.glob(self.basFilename + ".20*") This will make it a bit less dangerous for someone to copy a file as basename.save when they want to save the info for debugging later. (3) The midnight and weekly versions should recalculate the rolloverAt instead of adding interval. This will keep them from getting out of synch if there is a maintenance window or daylight savings time change. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=921318&group_id=5470 From noreply at sourceforge.net Thu Mar 25 14:32:29 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 14:32:36 2004 Subject: [Patches] [ python-Patches-923226 ] 'os' patch to speed up import (and python startup) Message-ID: Patches item #923226, was opened at 2004-03-25 17:18 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923226&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: 'os' patch to speed up import (and python startup) Initial Comment: This patch changes the _Environ subclass of UserDict to use dict instead, which avoids the import of UserDict, save 6-8ms on python startup with site.py (~4% improvement). I tested this on my MacOS X box, but since this code is really only used in Win32, someone should test it there. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-25 20:32 Message: Logged In: YES user_id=21627 The patch is incorrect. dict has no attribute data, and subclassing from dict involves many more changes than just changing __setitem__/ __getitem__. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923226&group_id=5470 From noreply at sourceforge.net Thu Mar 25 15:07:17 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 15:07:21 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:07 Message: Logged In: YES user_id=430343 Hopefully the last patch - whitespace screwed up in the last one. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 13:07 Message: Logged In: YES user_id=430343 Apparently the file didn't stick last time. Lets try it again. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 11:54 Message: Logged In: YES user_id=430343 I've attached a new patch which corrects the multiple import lock checks, and uses a global binding instead of the static mutable default arg hack.. :-) ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 10:34 Message: Logged In: YES user_id=430343 Yes, Jim's patch fixes the obvious problem...that's what I get for writing code after a long day at PyCon.. :-) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 15:39:47 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 16:02:47 2004 Subject: [Patches] [ python-Patches-923226 ] 'os' patch to speed up import (and python startup) Message-ID: Patches item #923226, was opened at 2004-03-25 11:18 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923226&group_id=5470 Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: 'os' patch to speed up import (and python startup) Initial Comment: This patch changes the _Environ subclass of UserDict to use dict instead, which avoids the import of UserDict, save 6-8ms on python startup with site.py (~4% improvement). I tested this on my MacOS X box, but since this code is really only used in Win32, someone should test it there. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:39 Message: Logged In: YES user_id=430343 Patch is broken, and not really worth fixing for the performance gain. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-03-25 14:32 Message: Logged In: YES user_id=21627 The patch is incorrect. dict has no attribute data, and subclassing from dict involves many more changes than just changing __setitem__/ __getitem__. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923226&group_id=5470 From noreply at sourceforge.net Thu Mar 25 15:37:40 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 16:02:50 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open >Resolution: Rejected >Priority: 1 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- >Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:37 Message: Logged In: YES user_id=430343 Arg - this only works in cvs. In the installed version it doesn't make any difference. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:07 Message: Logged In: YES user_id=430343 Hopefully the last patch - whitespace screwed up in the last one. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 13:07 Message: Logged In: YES user_id=430343 Apparently the file didn't stick last time. Lets try it again. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 11:54 Message: Logged In: YES user_id=430343 I've attached a new patch which corrects the multiple import lock checks, and uses a global binding instead of the static mutable default arg hack.. :-) ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 10:34 Message: Logged In: YES user_id=430343 Yes, Jim's patch fixes the obvious problem...that's what I get for writing code after a long day at PyCon.. :-) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 15:38:34 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 16:02:53 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Settings changed) made by mondragon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 >Status: Closed Resolution: Rejected Priority: 1 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:37 Message: Logged In: YES user_id=430343 Arg - this only works in cvs. In the installed version it doesn't make any difference. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:07 Message: Logged In: YES user_id=430343 Hopefully the last patch - whitespace screwed up in the last one. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 13:07 Message: Logged In: YES user_id=430343 Apparently the file didn't stick last time. Lets try it again. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 11:54 Message: Logged In: YES user_id=430343 I've attached a new patch which corrects the multiple import lock checks, and uses a global binding instead of the static mutable default arg hack.. :-) ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 10:34 Message: Logged In: YES user_id=430343 Yes, Jim's patch fixes the obvious problem...that's what I get for writing code after a long day at PyCon.. :-) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Thu Mar 25 22:17:02 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 22:17:09 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-25 19:17 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From noreply at sourceforge.net Thu Mar 25 22:53:35 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Thu Mar 25 22:53:42 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-26 03:17 Message generated for change (Comment added) made by phr You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-26 03:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From tlijmuuoec at ya.com Wed Mar 24 08:11:41 2004 From: tlijmuuoec at ya.com (Hilda Addison) Date: Fri Mar 26 06:18:50 2004 Subject: [Patches] archive Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040324/325c87ee/attachment.html From noreply at sourceforge.net Fri Mar 26 08:39:01 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 26 08:39:13 2004 Subject: [Patches] [ python-Patches-873224 ] Marshal clean-up Message-ID: Patches item #873224, was opened at 2004-01-08 19:29 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873224&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open >Resolution: Accepted Priority: 2 Submitted By: Armin Rigo (arigo) >Assigned to: Armin Rigo (arigo) Summary: Marshal clean-up Initial Comment: Reading random bytes with the marshal module can segfault the interpreter. Moreover, reading a truncated .pyc file can also successfully return a (corrupted) object instead of raising an error. Please review the attached patch if you consider this to be worth fixing. The patch also fixes a minor bug in import.c, which would sometimes write incomplete .pyc files in case of write error (and hence the truncated .pyc file problems could actually show up, in theory). ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-26 14:38 Message: Logged In: YES user_id=21627 The patch is fine, please apply. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-08 20:00 Message: Logged In: YES user_id=4771 Test cases: >>> marshal.loads('0') -> SystemError >>> marshal.loads('f') -> segfault >>> marshal.dumps(5L) 'l\x01\x00\x00\x00\x05\x00' >>> marshal.loads(_[:-1]) 65285L ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-08 19:55 Message: Logged In: YES user_id=6656 This patch results from my comments on IRC, so I'll take a look at it. Needs testcases! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873224&group_id=5470 From noreply at sourceforge.net Fri Mar 26 10:24:32 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 26 10:24:35 2004 Subject: [Patches] [ python-Patches-922115 ] PEP 292 reference implementation Message-ID: Patches item #922115, was opened at 2004-03-23 15:41 Message generated for change (Comment added) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922115&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Barry A. Warsaw (bwarsaw) Assigned to: Brett Cannon (bcannon) Summary: PEP 292 reference implementation Initial Comment: Here's a reference implementation for PEP 292, along with a few other things we talked about at PyCON II - creation of a top-level stringlib package - addition of stringlib/safedict.py which is a handy adjunct to PEP 292 I will also be checking in an updated PEP 292. Still to do: add test cases and documentation. I will do this in time for Python 2.4, if the PEP is accepted. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2004-03-26 07:24 Message: Logged In: YES user_id=357491 OK, the files (including a copy of string.py from Lib/) have been put in the sandbox in the 'string' directory. I will leave this patch open until something gets moved out os nondist/ into dist/ . ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922115&group_id=5470 From noreply at sourceforge.net Fri Mar 26 10:24:57 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 26 10:25:01 2004 Subject: [Patches] [ python-Patches-873224 ] Marshal clean-up Message-ID: Patches item #873224, was opened at 2004-01-08 18:29 Message generated for change (Settings changed) made by arigo You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873224&group_id=5470 Category: Core (C code) Group: Python 2.4 >Status: Closed Resolution: Accepted Priority: 2 Submitted By: Armin Rigo (arigo) Assigned to: Armin Rigo (arigo) Summary: Marshal clean-up Initial Comment: Reading random bytes with the marshal module can segfault the interpreter. Moreover, reading a truncated .pyc file can also successfully return a (corrupted) object instead of raising an error. Please review the attached patch if you consider this to be worth fixing. The patch also fixes a minor bug in import.c, which would sometimes write incomplete .pyc files in case of write error (and hence the truncated .pyc file problems could actually show up, in theory). ---------------------------------------------------------------------- >Comment By: Armin Rigo (arigo) Date: 2004-03-26 15:24 Message: Logged In: YES user_id=4771 Python/marshal.c rev 1.76 Python/import.c rev 2.229 Lib/test/test_marshal.py rev 1.6 Misformed .pyc files crashing Python is a bug that has been along for so long and people didn't complain, so I assume there is little point in bothering to back-port it to 2.3. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-03-26 13:38 Message: Logged In: YES user_id=21627 The patch is fine, please apply. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-08 19:00 Message: Logged In: YES user_id=4771 Test cases: >>> marshal.loads('0') -> SystemError >>> marshal.loads('f') -> segfault >>> marshal.dumps(5L) 'l\x01\x00\x00\x00\x05\x00' >>> marshal.loads(_[:-1]) 65285L ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-08 18:55 Message: Logged In: YES user_id=6656 This patch results from my comments on IRC, so I'll take a look at it. Needs testcases! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=873224&group_id=5470 From noreply at sourceforge.net Fri Mar 26 10:03:18 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Fri Mar 26 11:12:30 2004 Subject: [Patches] [ python-Patches-808120 ] Add --force-arch=ARCH to bdist_rpm.py Message-ID: Patches item #808120, was opened at 2003-09-17 16:28 Message generated for change (Comment added) made by fdrake You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=808120&group_id=5470 Category: Distutils and setup.py Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Tony Campbell (tcampbell) Assigned to: Nobody/Anonymous (nobody) Summary: Add --force-arch=ARCH to bdist_rpm.py Initial Comment: Allow a package builder to override the default architecture detection method employed by distutils by specifying --force-arch=ARCH on the command line; e.g., --force-arch=ppc This affects the BuildArch: tag in the resulting spec file. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2004-03-26 10:03 Message: Logged In: YES user_id=3066 Perhaps someone can point out documentation on the BuildArch: tag; I've googled and found both BuildArch: and BuildArchitectures:, but no reference documentation for either. A good pointer would be helpful. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=808120&group_id=5470 From ylvpwrhv at ya.com Fri Mar 26 11:13:37 2004 From: ylvpwrhv at ya.com (Ollie Tovar) Date: Fri Mar 26 13:12:16 2004 Subject: [Patches] excommunicate Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040326/cb84f655/attachment.html From n65metndt at fly.cc.fer.hr Thu Mar 25 10:21:53 2004 From: n65metndt at fly.cc.fer.hr (Brenton Bowling) Date: Sat Mar 27 00:43:13 2004 Subject: [Patches] A healthy balance sheet adds value to the stock price b mab qfuwd gxbwy Message-ID: **SRGE****SRGE****SRGE****SRGE****SRGE****SRGE** Market Watch News Flash Explosive short term trading profits in a new technology issue (Ticker: SRGE) are being predicted as many significant news releases indicate strong contractual revenues with major Telecom firms. SRGE has been successfully working with Telecommunications giants (with five million subscriber lines) over the last 4 years, but is now projecting "a banner expansion year with geometric growth in revenues" due largely to sales demands for their innovative patented products and expansion into International telecom markets. Surge Technologies, Inc. (SRGE) is a cutting-edge leader that designs, develops, manufactures, and markets superior patented outside plant electrical surge protection equipment for the telecommunications industry. The US sales projections for this market are $4 Billion annually, with this figure growing rapidly as the expansion of new HDSL and ADSL technologies permeate the industry. SRGE provides the Telecom industry with the highest quality "protection element" for complex digital switches. Protecting these Telecom switching devices is crucial to inclusive components that are sensitive to interruptions in voltage which can cause extensive network damage, thus negating costly and time-consuming repair and down-time. Major Telecoms require this protection throughout their network in order to prevent the hazards of harming personnel, damaging expensive equipment, and massive system failures. Opening Price: 1.00 10 Day Target: 2.30 1 Month Target: 4.50 Outstanding Shares: 16.5 million Est. Float: 2.2 million How many times have you seen issues explode but you couldn't get your hands on them or didn't have the right information in time? We are alerting you now to a special Company with a unique technology that is on the forefront of a breakout! We are excited about SRGE's technology and expansion as they prepare to ink deal after deal with Major US Telecoms in conjunction with dramatic increases in revenue for 2004 and 2005. SRGE has made phenomenal advancements but may be one of the few stocks left in this industry group that is unknown and undervalued, therefore a 300%-400% jump may wind up being conservative. -------------------------------------------------------- Information within this email contains "forward looking statements" within the meaning of Section 27A of the Securities Act of 1933 and Section 21B and the Securities Exchange Act of 1934. Any statements that express or involve discussions with respect to predictions, goals, expectations, beliefs, plans, projections, objectives, assumptions or future events or performance are not statements of historical fact and may be "forward looking statements". Forward looking statements are based upon expectations, estimates and projections, at the time the statements are made that involve a number of risks and uncertainties which could cause actual results or events to differ materially from those presently anticipated. Forward looking statements in this action may be identified through the use of words such as: "projects", "foresee", "expects", "estimates", "believes", "understands", "will", "anticipates", or that by statements indicating certain actions "may", "could", or "might" occur. All information provided within this email pertaining to investing, stocks, securities must be understood as information provided and not investment advice. Emerging Equity Alert advises all readers and subscribers to seek advice from a registered professional securities representative before deciding to trade in stocks featured within this email. None of the material within this report shall be construed as any kind of investment advice. In compliance with Section 17(b), we disclose the holdings of independently purchased shares of srge prior to the publication of this report. Be aware of an inherent conflict of interest resulting from such holdings due to our intent to profit from the liquidation of these shares. Shares may be sold at any time, even after positive statements have been made regarding the above company. wgyjbfi tgd x o gwsvxg voks dqjpytae rqaqkezj From noreply at sourceforge.net Sat Mar 27 01:45:33 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 27 01:45:40 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-25 19:17 Message generated for change (Comment added) made by trevp You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) >Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- >Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-25 19:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From noreply at sourceforge.net Sat Mar 27 01:45:54 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 27 01:45:58 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-25 19:17 Message generated for change (Comment added) made by trevp You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- >Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-25 19:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From noreply at sourceforge.net Sat Mar 27 01:57:57 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 27 01:58:02 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-26 03:17 Message generated for change (Comment added) made by phr You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-27 06:57 Message: Logged In: YES user_id=72053 How about just punting signed conversion. I don't think two's complement makes much sense for arbitrary precision longs. Have some separate representation for negative longs if needed. If you call hex() on a large negative number, you get a hex string with a leading minus sign. For base 256, you can't reserve a char like that, so I guess you have to just throw an error if someone tries to convert a negative long to a string. If you want a representation for signed longs, ASN1 DER is probably an ok choice. I agree with Guido that the binascii module is a good place to put such a function. Twos complement can work if you specify a fixed precision, but that sure complicates what this started out as. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-27 06:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-27 06:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-26 03:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From noreply at sourceforge.net Sat Mar 27 02:51:55 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 27 02:52:02 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-25 19:17 Message generated for change (Comment added) made by trevp You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- >Comment By: Trevor Perrin (trevp) Date: 2004-03-26 23:51 Message: Logged In: YES user_id=973611 I think 2's complement makes good sense for arbitrary precision longs. This is how OpenSSL and GMP handle them. It's also how the ASN.1 BER/DER encodings handle integers: these encodings just prepend tag and length fields to the big- endian 2's complement value. I.e.: If you want to extract RSA public values from an X.509 certificate, they'll be in 2's complement (well, they'll always be positive... but they'll have an extra zero byte if necessary). Since the functionality for 2's complement is already in the C code it's easy to expose through a patch. So I'm still in favor of presenting it. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-26 22:57 Message: Logged In: YES user_id=72053 How about just punting signed conversion. I don't think two's complement makes much sense for arbitrary precision longs. Have some separate representation for negative longs if needed. If you call hex() on a large negative number, you get a hex string with a leading minus sign. For base 256, you can't reserve a char like that, so I guess you have to just throw an error if someone tries to convert a negative long to a string. If you want a representation for signed longs, ASN1 DER is probably an ok choice. I agree with Guido that the binascii module is a good place to put such a function. Twos complement can work if you specify a fixed precision, but that sure complicates what this started out as. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-25 19:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From noreply at sourceforge.net Sat Mar 27 10:56:45 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 27 10:57:02 2004 Subject: [Patches] [ python-Patches-872326 ] generator expression implementation Message-ID: Patches item #872326, was opened at 2004-01-07 21:10 Message generated for change (Comment added) made by jiwon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 Category: Parser/Compiler Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jiwon Seo (jiwon) Assigned to: Raymond Hettinger (rhettinger) Summary: generator expression implementation Initial Comment: Since I was interested in pep 289(generator expression), I dabbled with it, and implemented a working version of it. I'm not sure if I did it right, but maybe someone who knows better can fix it right. 1. Grammar has two changes, which is a. atom: '(' [testlist] ')' | '[' [listmaker] ']' | ... changes to atom: '(' [testgenexpr] ')' | '[' [listmaker] ']' | ... where testgenexpr defines like this. testgenexpr: test ( gen_for | (',' test)* [','] ) b. argument: [test '='] test changes to argument: [test '='] test [gen_for] (gen_for, gen_if, gen_iter is similar to list_for, list_if, list_iter respectively.) 2. Instead of changing rule of arglist in Grammar to accept generator expression, I changed argument rule like 1.b. This means Grammar accepts generator expression without parenthesis in function call even there are several arguments, like reduce(operator.add, (x for x in range(10))) This is against what pep requires, so I made parsermodule.c and compile.c to catch that and throw error message when there's more than one argument in a function. The reason I did it that way is to go around a problem of ambiguity in the grammar by adding generator expression to arglist. 3. I implemented generator expression as equivalent to a call to a function which has variable capturing code and generator code. For example, x = 1 g = (x for i in range(10)) is equivalent to x = 1 def __gen_wrapper(): _[x] = x # variable capture here def __gen(): for i in range(10): yield _[x] return __gen() g = __gen_wrapper() 4. Since I implemented generator expression equivalent to a function call, symbol table should enter new scope when it meets generator expression. So I ended up with adding following code to symtable.c PyObject * PySymtableEntry_New(struct symtable *st, char *name, int type, int lineno) { ... switch (type) { case funcdef: case lambdef: ! case testgenexpr: /* generator expression */ ! case argument: /* generator expression */ ste->ste_type = TYPE_FUNCTION; break; ... so that generator expression can be dealt as function type. This is kind of stupid, but I couldn't find other easy for it, so I just left it like that. 5. this patch does not include diff of src/Lib/symbol.py, so you must run python Lib/symbol.py to get it right. ---------------------------------------------------------------------- >Comment By: Jiwon Seo (jiwon) Date: 2004-03-28 00:56 Message: Logged In: YES user_id=595483 I'm submitting two versions now. One is variable-capture version(but without any iterable pre-computation), and the other is lazy-binding version. I'll be in military camp for about 1 month(4/6~5/2) so I will not be able to fix/patch anything for the period, thus I'm submitting this in now. :) If I have time, and necessity arises (before 4/6), I'll also add outmost-iterable-precomputation version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-03-23 08:00 Message: Logged In: YES user_id=6380 Sorry, I meant the first, not the last. I was confused about how the 'for' clauses are nested, but the outermost one is the first. So the nesting remark is probably just confusing and wrong; ignore it. What I meant is: (f(x,y) for x in A() for y in B(x)) should IMO precompute A(), but not B(x). I guess the equivalent generator function would be: def __gen(__outer__=A(), f=f, B=B): for x in __outer__: for y in B(x): yield f(x,y) In general the value of every free variable used anywhere except in the outer expression should be captured; the *value* of the outer expression should be captured. This should give the least surprising semantics in a variaty of use cases. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2004-03-23 04:39 Message: Logged In: YES user_id=6380 Only the outermost (last) iterable should be precomputed. While (f(x,y) for x in A(y) for y in B) is not the same as ((f(x,y) for x in A(y)) for y in B) I think the scoping should be done similarly. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-22 10:57 Message: Logged In: YES user_id=595483 Ah... Maybe we need to drop precomputation of iterables. I'll post it on the mailing list so that people can consider about that. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-03-22 08:40 Message: Logged In: YES user_id=4771 Oh ho. Am I right in fearing that the idea of precomputing the iterables was broken in the first place? We just cannot do this. The things we are looping over may depend on other stuff from the generator expression itself. Example: >>> [x for l in [[1,2],[3,4]] for x in l] [1, 2, 3, 4] >>> (y for m in [[1,2],[3,4]] for y in m) NameError: name 'm' is not defined ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:55 Message: Logged In: YES user_id=55188 This inconsistency may not be fixed by list(iter()) workaround, either. >>> def counter(): ... counter.count += 1 ... return range(counter.count) ... >>> counter.count = 0 >>> [y for x in 'abc' for y in counter()] [0, 0, 1, 0, 1, 2] >>> counter.count = 0 >>> list(y for x in 'abc' for y in counter()) [0, 0, 0] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:42 Message: Logged In: YES user_id=55188 Aah. But I'm not insisting on that because it's incompatible even with plain nested for-loops. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-22 02:38 Message: Logged In: YES user_id=55188 Agreed. I'd prefer the current implementation's behavor rather than defined in PEP289. Yes, It's incompatible with list comprehensions but generator expressions are already quite different enough from it. :) How about to change PEP289's genexpr semantics to this? g = (x**2 for x in range(10)) print g.next() is equivalent to: def __gen(_i1): for x in _i1: yield x**2 g = __gen(range(10)) print g.next() ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-22 02:28 Message: Logged In: YES user_id=80475 You need a solution other than list(). A key to the success of genexps is to never to eat memory by expanding an iterator into a container. For behavior questions, your reference point is the list comprehension. list(genexp) should always produce the same result as a list comprehension. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-22 01:59 Message: Logged In: YES user_id=595483 Hi, quiver. I don't think we can easily go around this problem if we have to capture iterators in generator expression. If you run following, you'll know what I mean. >>> a = iter("abcd") >>> b = iter("abcd") >>> [(x, y) for x in a for y in b] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] I think one possible solution could be, instead of passing evaluations of iterators in generator expression, make a list from iterator and, then pass it as argument. For instance, g = (x for x in iter("abcd")) will be equivalent to, def __gen(_[1]): for x in _[1]: yield x g = __gen(list(iter("abcd"))) # see 'list' - instead of g = __gen(iter("abcd")) . I'm not sure if I'm in a position to decide to do that way or not. If the current reviewer (rhettinger) approves it, I'll do that way. Or else, I think I will post it on the mailing list. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-03-19 22:37 Message: Logged In: YES user_id=671362 Hi, Jiwon. This is another bug candidate. If you use genexp with iterators, it doesn't work well. # test Perky's previous post using iterators >>> list((x,y) for x in iter('abcd') for y in iter('abcd')) [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Thanks. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 17:45 Message: Logged In: YES user_id=595483 To fix the bug that perky picked out, I hand-made ast.py instead of using auto-generated code. But, still I don't understand why Tools/compiler/regrtest didn't tell me about it. (Maybe I didn't look at the output carefully enough.) Ah, and it would be nice if somebody tell me how to run test_grammar.py(only) with python-compiler package. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 15:28 Message: Logged In: YES user_id=55188 The compiler package patch has some problems in its compiler/ast.py currently. jiwon said he regenerated it using Tools/compiler/astgen.py. But it made some incompatibilities against ast.py on current CVS. For example, class Expression. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-17 15:21 Message: Logged In: YES user_id=80475 I'll give it a second review. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-17 14:37 Message: Logged In: YES user_id=55188 Okay. Here's my draft for documentation. Any review/fix/enhance is very welcome! I think it's ready to putting it into CVS now. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-03-17 01:24 Message: Logged In: YES user_id=595483 ah, the following bug was due to the patch I uploaded 2004-02-04 15:13. In order to get an error instantly from an expression like "g=(x for x in None)", I made it equivalent to, def __gen(_[1]): for x in _[1]: yield x g=__gen(iter(None)) ^^^^ But when there is two or more iterator expression of same symbol(or string), that patch produces error, because currently, "((x, y) for x in 'abcd' for y in 'abcd') " is equivalent to, def __gen(_[1]): for x in _[1]: for y in _[1]: yield (x,y) g = __gen(iter("abcd")) # passing only one parameter I could make it pass every iterator expressions respectively even when they are same symbol(or string, ...), but for now, I just dropped that patch (patch of 2004-02-04) since that's the simplest thing now. If somebody says getting instance error for cases like "(x for x in None)" is important, I'll make another patch for it. Btw, added more test case that covers what perky mentioned. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-16 13:16 Message: Logged In: YES user_id=55188 Another bug in the implementation. >>> list((x, y) for x in 'abcd' for y in 'abcd') [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd')] Expected behavior may be: >>> [(x, y) for x in 'abcd' for y in 'abcd'] [('a', 'a'), ('a', 'b'), ('a', 'c'), ('a', 'd'), ('b', 'a'), ('b', 'b'), ('b', 'c'), ('b', 'd'), ('c', 'a'), ('c', 'b'), ('c', 'c'), ('c', 'd'), ('d', 'a'), ('d', 'b'), ('d', 'c'), ('d', 'd')] ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-03-14 11:59 Message: Logged In: YES user_id=55188 I'm writing docs for tutorial and language reference. It'll be completed in a day or two. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2004-03-14 06:17 Message: Logged In: YES user_id=80475 Any recent progress? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-24 01:34 Message: Logged In: YES user_id=595483 Whoa! I finally patched compiler package in pure python. (I was a bit busy recently :) I've run regression test with 'Tools/compiler/regrtest.py' before I submit this patch, and there was one failure (which is test_dis.py). I'm not sure if that's a problem, so I'm just submitting this in. Now, I think I'll refactor things a bit! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-04 22:53 Message: Logged In: YES user_id=55188 As it mentioned in PEP, even listcomp's leaking of loop value will be dropped in Python 3. I'm sorry but I don't see that the usage is solid in the future. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-04 19:45 Message: Logged In: YES user_id=671362 What about this code? In the currently implementation, loop variables inside a list comprehension is not visible outside if you use it inside a generator expression. For example: >>> (a*b for a in [b for b in range(5)]) Traceback (most recent call last): File "", line 1, in ? NameError: name 'b' is not defined Its list comprehension counterpart is: >>> [a*b for a in [b for b in range(5)]] [0, 4, 8, 12, 16] ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-04 15:13 Message: Logged In: YES user_id=595483 Again, I fixed the patch so that it can get error from '(x for x in None)' immediately. (upto now, error does not occur until generator expression is evaluated) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-03 20:46 Message: Logged In: YES user_id=4771 After thinking a bit more on the issue, note that the generator version of your example is equivalent to: def g(): for y in range(3): yield lambda x: x+y which means that it can suffer from the same problem as the first example, but more subtly (and even more confusingly): for f in g(): print f(0) # 0, 1, 2 for f in list(g()): print f(0) # 2, 2, 2 This is because due to Python's nested scope rules the above generator is equivalent to: def g(): result = lambda x: x+y for y in range(3): yield result at which point I think it gets pretty confusing... ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-03 18:07 Message: Logged In: YES user_id=671362 Thanks, Arigo and Perky. Hmm, maybe I should read PEP and the thread about namespace more carefully, not just skim the surface. Anyway, PEP has not been updated since last October, so I think it's good time to merge recent progress of genexpr and update PEP-289. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-02-02 21:58 Message: Logged In: YES user_id=4771 The behavior is indeed the one described by the PEP but it is still surprising. As far as I'm concerned it is yet another reason why free variable bindings ("nested scopes") are done wrong in Python currently :-( If you use the older trick "lambda x,y=y:x+y" instead, then both cases will agree (with the sensible result in my opinion). A few more issues and I'll start a campain for definition-time bindings of free variables :-( ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-02-02 21:37 Message: Logged In: YES user_id=55188 I think it's right for namespace difference between listcomp and genexpr. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-02-02 20:47 Message: Logged In: YES user_id=671362 I am not sure if I should call this a bug, but here it goes: >>> lst = [lambda x:x+y for y in range(3)] >>> for f in lst:print f(0) ... 2 2 2 >>> gen = (lambda x:x+y for y in range(3)) >>> for f in gen:print f(0) ... 0 1 2 Is this the intended behavior? ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-02-02 17:29 Message: Logged In: YES user_id=595483 ok. I fixed another bug reported by perky, and added related test to test_grammar.py. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-30 14:09 Message: Logged In: YES user_id=55188 Yet another Fatal Python error; >>> (a for a in (b for b in (a for a in 'xxx' if True) if False) if True) lookup 'True' in ? 3 -1 freevars of : ('True',) Fatal Python error: com_make_closure() zsh: abort (core dumped) ./python # applied patch as of 2004-01-30 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-30 12:01 Message: Logged In: YES user_id=595483 ok, I've fixed the bug quiver commented, and added related test code to test_grammar.py. ---------------------------------------------------------------------- Comment By: George Yoshida (quiver) Date: 2004-01-29 20:48 Message: Logged In: YES user_id=671362 yet another Fatal Python error; >>> (a for a in (b for b in (0,1))) >>> (a for a in (b for b in range(2))) Fatal Python error: unknown scope for range in ?(0) in symbols: {'range': 8} locals: {} globals: {} Aborted # applied patch as of 2004-01-27 ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-27 16:44 Message: Logged In: YES user_id=595483 I fixed the patch for the bug that arigo mentioned, and for what perky mentioned - PyList_GetSlice error handling - . now, I'll tackle python compiler package :) ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-27 00:15 Message: Logged In: YES user_id=4771 >>> (a for d in a) Fatal Python error: unknown scope for a in (1) in symbols: {'a': 2048, '_[1]': 4, 'd': 2} locals: {'_[1]': 0, 'd': 1} globals: {} ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 15:17 Message: Logged In: YES user_id=55188 Please ignore the previous comment. It was a result from an old revision of patches. It works on the recent. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-26 08:44 Message: Logged In: YES user_id=55188 BTW, does unavaliability of yield (genexpr) and return (genexpr) an intended behavior? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-01-22 03:13 Message: Logged In: YES user_id=6656 Documentation would be nice! ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-21 23:14 Message: Logged In: YES user_id=55188 Okay. My review is done. The revised patch "gexp.diff" looks fine for me. There're few minor tweaks still needed to get into CVS. - Some error exit cases are ignored. You need to provide safe exit paths for MemoryError. (eg. PyList_GetSlice usage of Python/ compiler.c) - A patch for 'compiler' pure python package. (there's an old implementation on SF #795947) ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-19 20:10 Message: Logged In: YES user_id=595483 ok. I modified the patch so that it evaluates iterator expr in generator expression creation time. That means, g = (x for x in range(10)) is equivalent to def __gen(iter1): for x in iter1: yield x g = __gen(range(10)) so, evalution of range(10) is not deferred anymore. I also changed testgenexpr to testlist_gexp in Grammar/Grammar, since Hye-Shik Chang advised as such. I'm willing to take any advice about this patch, so please do. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-12 12:26 Message: Logged In: YES user_id=595483 ok. I've implemented capturing variables part as arigo suggested. File genexpr-capture-vars-in-args.diff is that. If you look at the code, you'll find that the code for generator expression is much shorter, and there's lesser modification in the existing code. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-09 10:49 Message: Logged In: YES user_id=595483 What arigo wrote sounds reasonable, and not very difficult to implement. I'll try to do that way. ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-09 03:50 Message: Logged In: YES user_id=4771 We may not need two levels of nested anonymous functions. It seems to me that the following equivalent code would do, because it captures the variable in an argument instead of via nested scopes: x = 1 def __gen(x): for i in range(10): yield x g = __gen(x) I don't know though if this is easy to implement in compile.c. Alternatively: x = 1 def __gen(x=x): for i in range(10): yield x g = __gen() ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-08 14:44 Message: Logged In: YES user_id=55188 Okay. I verified that basic characteristics mentioned on PEP are working. I started to review the implementation. ---------------------------------------------------------------------- Comment By: Jiwon Seo (jiwon) Date: 2004-01-08 13:50 Message: Logged In: YES user_id=595483 I added diff of Lib/symbol.py, so no need to run python Lib/symbol.py now. ---------------------------------------------------------------------- Comment By: Hye-Shik Chang (perky) Date: 2004-01-07 21:25 Message: Logged In: YES user_id=55188 Assigned to me. The originator is my friend and I have much interest on this. :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=872326&group_id=5470 From noreply at sourceforge.net Sat Mar 27 11:03:00 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sat Mar 27 11:03:04 2004 Subject: [Patches] [ python-Patches-924497 ] Updates to the Misc/RPM spec file. Message-ID: Patches item #924497, was opened at 2004-03-27 16:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=924497&group_id=5470 Category: Build Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: Sean Reifschneider (jafo) Assigned to: Nobody/Anonymous (nobody) Summary: Updates to the Misc/RPM spec file. Initial Comment: Because of continued problems with getting all the #!/usr/local/bin/python references out, I'm using a "find" to find the files that need to be fixed, instead of including a static list. Sean ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=924497&group_id=5470 From ZSZDOGCCY at mednet.com.br Fri Mar 26 16:53:02 2004 From: ZSZDOGCCY at mednet.com.br (Coleman Levine) Date: Sat Mar 27 11:15:45 2004 Subject: [Patches] use google adwords together with affiliate program to make big profits Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040326/be9d3518/attachment.html From noreply at sourceforge.net Sun Mar 28 01:40:13 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 28 01:40:16 2004 Subject: [Patches] [ python-Patches-924771 ] work around to compile \r\n file Message-ID: Patches item #924771, was opened at 2004-03-28 15:40 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=924771&group_id=5470 Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: George Yoshida (quiver) Assigned to: Nobody/Anonymous (nobody) Summary: work around to compile \r\n file Initial Comment: On Unix-like systems, built-in function compile cannot compile files that don't use linefeed as an EOL. But there are several codes in the library which assume that input files use valid EOL characters. So when they come across \r\n(or \r) on Unix environments, compile causes the parser to raise a SyntaxError. For example: >>> list(file('a.py')) ['\r\n'] >>> import trace >>> trace.find_executable_linenos('a.py') Traceback (most recent call last): File "", line 1, in ? File "/usr/lib/python2.3/trace.py", line 389, in find_executable_linenos code = compile(prog, filename, "exec") File "a.py", line 1 ^ SyntaxError: invalid syntax Lib/py_compile.py opens files with 'U' option to handle file format differences and I think this is the way to go. There is one drawback. When Python is configured without universal newline support, this approach doesn't work :-( ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=924771&group_id=5470 From xmkkyqkuylv at msn.com Mon Mar 29 07:06:40 2004 From: xmkkyqkuylv at msn.com (Seth Corbin) Date: Sun Mar 28 16:04:52 2004 Subject: [Patches] pornstar size Message-ID: Drop the HAMMER on the next bitch you Nail.... http://pulaski.abonepha.com/vp5 Click here if you do not wish to be invited again: http://fightnottobesingle.com/remove/?oc=xxxxyyyy From noreply at sourceforge.net Sun Mar 28 19:54:56 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 28 19:55:06 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-25 19:17 Message generated for change (Comment added) made by trevp You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- >Comment By: Trevor Perrin (trevp) Date: 2004-03-28 16:54 Message: Logged In: YES user_id=973611 My last comment was wrong: GMP's raw input/output format uses big-endian positive values, with the sign bit stored separately. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 23:51 Message: Logged In: YES user_id=973611 I think 2's complement makes good sense for arbitrary precision longs. This is how OpenSSL and GMP handle them. It's also how the ASN.1 BER/DER encodings handle integers: these encodings just prepend tag and length fields to the big- endian 2's complement value. I.e.: If you want to extract RSA public values from an X.509 certificate, they'll be in 2's complement (well, they'll always be positive... but they'll have an extra zero byte if necessary). Since the functionality for 2's complement is already in the C code it's easy to expose through a patch. So I'm still in favor of presenting it. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-26 22:57 Message: Logged In: YES user_id=72053 How about just punting signed conversion. I don't think two's complement makes much sense for arbitrary precision longs. Have some separate representation for negative longs if needed. If you call hex() on a large negative number, you get a hex string with a leading minus sign. For base 256, you can't reserve a char like that, so I guess you have to just throw an error if someone tries to convert a negative long to a string. If you want a representation for signed longs, ASN1 DER is probably an ok choice. I agree with Guido that the binascii module is a good place to put such a function. Twos complement can work if you specify a fixed precision, but that sure complicates what this started out as. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-25 19:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From noreply at sourceforge.net Sun Mar 28 19:55:29 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Sun Mar 28 19:55:33 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-25 19:17 Message generated for change (Comment added) made by trevp You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- >Comment By: Trevor Perrin (trevp) Date: 2004-03-28 16:55 Message: Logged In: YES user_id=973611 My last comment was wrong: GMP's raw input/output format uses big-endian positive values, with the sign bit stored separately. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-28 16:54 Message: Logged In: YES user_id=973611 My last comment was wrong: GMP's raw input/output format uses big-endian positive values, with the sign bit stored separately. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 23:51 Message: Logged In: YES user_id=973611 I think 2's complement makes good sense for arbitrary precision longs. This is how OpenSSL and GMP handle them. It's also how the ASN.1 BER/DER encodings handle integers: these encodings just prepend tag and length fields to the big- endian 2's complement value. I.e.: If you want to extract RSA public values from an X.509 certificate, they'll be in 2's complement (well, they'll always be positive... but they'll have an extra zero byte if necessary). Since the functionality for 2's complement is already in the C code it's easy to expose through a patch. So I'm still in favor of presenting it. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-26 22:57 Message: Logged In: YES user_id=72053 How about just punting signed conversion. I don't think two's complement makes much sense for arbitrary precision longs. Have some separate representation for negative longs if needed. If you call hex() on a large negative number, you get a hex string with a leading minus sign. For base 256, you can't reserve a char like that, so I guess you have to just throw an error if someone tries to convert a negative long to a string. If you want a representation for signed longs, ASN1 DER is probably an ok choice. I agree with Guido that the binascii module is a good place to put such a function. Twos complement can work if you specify a fixed precision, but that sure complicates what this started out as. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-25 19:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From Felicia.pluperfect at yahoo.com Sun Mar 28 11:33:19 2004 From: Felicia.pluperfect at yahoo.com (Felicia Sams) Date: Mon Mar 29 00:33:31 2004 Subject: [Patches] Bachelors, Masters, MBA and/or Doctorate (PhD) pompadour Message-ID: <299914351976.18300@Felicia.intonate@yahoo.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040328/ee9407e8/attachment.html From lcyg9hdki at ya.com Sun Mar 28 21:24:47 2004 From: lcyg9hdki at ya.com (Carol Ayers) Date: Mon Mar 29 06:37:52 2004 Subject: [Patches] cheap software deals swag Message-ID: <7dmr717$d8tje8k7m@hj7hk67jj> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040329/0a49e370/attachment.html From SBLHIWAASMV at buag.co.at Sun Mar 28 17:29:01 2004 From: SBLHIWAASMV at buag.co.at (Jacqueline Ford) Date: Mon Mar 29 14:43:37 2004 Subject: [Patches] what could you accomplish with a degree? Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20040329/29f0e5f7/attachment.html From noreply at sourceforge.net Tue Mar 30 02:10:54 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 30 02:11:03 2004 Subject: [Patches] [ python-Patches-923643 ] long <-> byte-string conversion Message-ID: Patches item #923643, was opened at 2004-03-25 19:17 Message generated for change (Comment added) made by josiahcarlson You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Trevor Perrin (trevp) Assigned to: Nobody/Anonymous (nobody) Summary: long <-> byte-string conversion Initial Comment: Sometimes you want to turn a long into a byte-string, or vice-versa. This is useful in cryptographic protocols, and probably other network protocols where you need to exchange large integers. In 2.4, you can handle unsigned longs with this: def stringToLong(s): return long(binascii.hexlify(s), 16) def longToString(n): return binascii.unhexlify("%x" % n) However, these functions are slower than they need to be, they're kinda kludgey, and they don't handle negative values. So here's a proposal: def stringToLong(s): return long(s, 256) def longToString(n): return n.tostring() These functions operate on big-endian, 2's-complement byte-strings. If the value is positive but has its most- significant-bit set, an extra zero-byte will be prepended. This is the same way OpenSSL and (I think) GMP handle signed numbers. These functions are ~5x faster than the earlier ones, they're cleaner, and they work with negative numbers. If you only want to deal with unsigned positive numbers, you'll have to do some adjustments: def stringToLong(s): return long('\0'+s, 256) def longToString(n): s = n.tostring() if s[0] == '\0' and s != '\0': s = s[1:] return s That's not ideal, but it seems better than any interface change I could think of. Anyways, the patch adds this to longs. It should be added to ints too, and I guess it needs tests etc.. I can help with that, if the basic idea is acceptable. Trevor ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2004-03-29 23:10 Message: Logged In: YES user_id=341410 I'm curious to know if anyone would object to optional minimum or maximum or both arguments, or even some additional methods that would result in a potentially constrained string output from long.tostring()? If I were to split the functionality into three methods, they would be as follows... def atleast(long, atl): if atl < 0: raise TypeError("atleast requires a positive integer for a minimum length") a = long.tostring() la = len(a) return (atl-la)*'\o' + a def atmost(long, atm): if atm < 0: raise TypeError("atleast requires a positive integer for a minimum length") a = long.tostring() la = len(a) return a[:atm] def constrained(long, atl, atm): if atm < atl: raise TypeError("constrained requires that the maximum length be larger than the minimum length") if atl < 0 or atm < 0: raise TypeError("constrained requires that both arguments are positive") a = long.tostring() la = len(a) return ((atl-la)*'\o' + a)[:atm] I personally would find use for the above, would anyone else have use for it? ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-28 16:55 Message: Logged In: YES user_id=973611 My last comment was wrong: GMP's raw input/output format uses big-endian positive values, with the sign bit stored separately. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-28 16:54 Message: Logged In: YES user_id=973611 My last comment was wrong: GMP's raw input/output format uses big-endian positive values, with the sign bit stored separately. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 23:51 Message: Logged In: YES user_id=973611 I think 2's complement makes good sense for arbitrary precision longs. This is how OpenSSL and GMP handle them. It's also how the ASN.1 BER/DER encodings handle integers: these encodings just prepend tag and length fields to the big- endian 2's complement value. I.e.: If you want to extract RSA public values from an X.509 certificate, they'll be in 2's complement (well, they'll always be positive... but they'll have an extra zero byte if necessary). Since the functionality for 2's complement is already in the C code it's easy to expose through a patch. So I'm still in favor of presenting it. ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-26 22:57 Message: Logged In: YES user_id=72053 How about just punting signed conversion. I don't think two's complement makes much sense for arbitrary precision longs. Have some separate representation for negative longs if needed. If you call hex() on a large negative number, you get a hex string with a leading minus sign. For base 256, you can't reserve a char like that, so I guess you have to just throw an error if someone tries to convert a negative long to a string. If you want a representation for signed longs, ASN1 DER is probably an ok choice. I agree with Guido that the binascii module is a good place to put such a function. Twos complement can work if you specify a fixed precision, but that sure complicates what this started out as. ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: Trevor Perrin (trevp) Date: 2004-03-26 22:45 Message: Logged In: YES user_id=973611 You're right, we should support unsigned strings somehow. Adding another argument to the int() and long() constructors would be messy, though. How about: n = long(s, 256) #unsigned n = long(s, -256) #signed n.tounsignedstring() n.tosignedstring() The "-256" thing is a hack, I admit.. but it kinda grows on you, if you stare it at awhile :-)... ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2004-03-25 19:53 Message: Logged In: YES user_id=72053 I think those funcs should take an optional extra arg to say you want unsigned. That's cleaner than prepending '0'. In cryptography you usually do want unsigned. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=923643&group_id=5470 From noreply at sourceforge.net Tue Mar 30 05:55:25 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 30 05:55:36 2004 Subject: [Patches] [ python-Patches-925932 ] struct.pack() on 64bit architectures Message-ID: Patches item #925932, was opened at 2004-03-30 12:55 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 Category: Tests Group: None Status: Open Resolution: None Priority: 5 Submitted By: Aleksander Piotrowski (pelotas) Assigned to: Nobody/Anonymous (nobody) Summary: struct.pack() on 64bit architectures Initial Comment: I'm running python on OpenBSD/sparc64 (SUN Ultra 10). On this machine struct.pack() gives me: >>> struct.pack('l', 1) '\x00\x00\x00\x00\x00\x00\x00\x01' >>> struct.pack('i', 1) '\x00\x00\x00\x01' On i386 box I have: >>> struct.pack('l', 1) '\x01\x00\x00\x00' >>> struct.pack('i', 1) '\x01\x00\x00\x00' Because of this, OpenBSD port uses attached patch. I guess that this is generic problem (not OpenBSD specific) and every 64bit platform would suffer. Am I right? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 From noreply at sourceforge.net Tue Mar 30 09:56:20 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 30 09:56:26 2004 Subject: [Patches] [ python-Patches-922881 ] Patch for substantial startup time reduction Message-ID: Patches item #922881, was opened at 2004-03-24 22:35 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Closed Resolution: Rejected Priority: 1 Submitted By: Nick Bastin (mondragon) Assigned to: Nobody/Anonymous (nobody) Summary: Patch for substantial startup time reduction Initial Comment: This patch puts the delayed import of linecache back into warnings.py, but protects it with a check against the import lock being held by someone else. This results in regaining 50% of the launch speed that we lost moving from 2.2.2 to 2.3. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-30 09:56 Message: Logged In: YES user_id=764593 Does that still make it worth applying for 2.4? It isn't really a good 2.3 candidate anyhow, since "slow" isn't a bug. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:37 Message: Logged In: YES user_id=430343 Arg - this only works in cvs. In the installed version it doesn't make any difference. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 15:07 Message: Logged In: YES user_id=430343 Hopefully the last patch - whitespace screwed up in the last one. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 13:07 Message: Logged In: YES user_id=430343 Apparently the file didn't stick last time. Lets try it again. ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 11:54 Message: Logged In: YES user_id=430343 I've attached a new patch which corrects the multiple import lock checks, and uses a global binding instead of the static mutable default arg hack.. :-) ---------------------------------------------------------------------- Comment By: Nick Bastin (mondragon) Date: 2004-03-25 10:34 Message: Logged In: YES user_id=430343 Yes, Jim's patch fixes the obvious problem...that's what I get for writing code after a long day at PyCon.. :-) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:13 Message: Logged In: YES user_id=764593 It isn't letting me a attach a file just now, so I'll paste. # Assumes that imp is imported at the top instead of # linecache, as in the original patch. def formatwarning(message, category, filename, lineno, linec = []): """Function to format a warning the standard way.""" s = "%s:%s: %s: %s\n" % (filename, lineno, category. __name__, message) if not linec: if imp.lock_held(): # Somebody else is holding the import lock. To avoid # a deadlock we just return the string so in worst # case the user can look it up themselves. Sorry. return s else: import linecache linec.append(linecache.getline) line = linec[0](filename, lineno).strip() if line: s = s + " " + line + "\n" return s ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2004-03-25 10:10 Message: Logged In: YES user_id=764593 You only need to import linecache once; after that, it shouldn't matter whether the lock is held. You can track whether it was loaded with either a global or a mutable default value, but I'm not sure how unacceptable the style would be. The warningupdate files shows a formatwarning with the mutable default variant. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=922881&group_id=5470 From noreply at sourceforge.net Tue Mar 30 10:57:22 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 30 11:02:30 2004 Subject: [Patches] [ python-Patches-925932 ] struct.pack() on 64bit architectures Message-ID: Patches item #925932, was opened at 2004-03-30 05:55 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 Category: Tests Group: None Status: Open Resolution: None Priority: 5 Submitted By: Aleksander Piotrowski (pelotas) Assigned to: Nobody/Anonymous (nobody) Summary: struct.pack() on 64bit architectures Initial Comment: I'm running python on OpenBSD/sparc64 (SUN Ultra 10). On this machine struct.pack() gives me: >>> struct.pack('l', 1) '\x00\x00\x00\x00\x00\x00\x00\x01' >>> struct.pack('i', 1) '\x00\x00\x00\x01' On i386 box I have: >>> struct.pack('l', 1) '\x01\x00\x00\x00' >>> struct.pack('i', 1) '\x01\x00\x00\x00' Because of this, OpenBSD port uses attached patch. I guess that this is generic problem (not OpenBSD specific) and every 64bit platform would suffer. Am I right? ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2004-03-30 10:57 Message: Logged In: YES user_id=31435 For a Unix-head reviewer: the patch is to test_fcntl.py, which uses a native struct.pack() to build a lockdata argument for fcntl. On the OP's box, it constructs stuff of the wrong size, due to "l" format codes producing 8-byte thingies. The patch changes the "l" codes to "i" on the OP's platform. I wonder whether we couldn't instead use "i" codes on all BSD- ish platforms -- on 32-bit boxes, "i" is synonymous with "l", while on 64-bit boxes it looks like "l" is wrong but "i" is correct. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 From noreply at sourceforge.net Tue Mar 30 14:46:06 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 30 15:03:28 2004 Subject: [Patches] [ python-Patches-926209 ] Patch to setup.py to run on x86_64 Linux Message-ID: Patches item #926209, was opened at 2004-03-30 14:46 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926209&group_id=5470 Category: Build Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Christopher Petrilli (petrilli) Assigned to: Nobody/Anonymous (nobody) Summary: Patch to setup.py to run on x86_64 Linux Initial Comment: Bug ID#924333 Python doesn't auto-detect things correctly because of differing library locations on x86_64 Linux distributions (/usr/lib64). This patch just adds that to the setup.py file. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926209&group_id=5470 From noreply at sourceforge.net Tue Mar 30 19:51:17 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Tue Mar 30 19:51:21 2004 Subject: [Patches] [ python-Patches-926375 ] unichr(wide) error on ucs2 build Message-ID: Patches item #926375, was opened at 2004-03-31 09:51 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926375&group_id=5470 Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Hye-Shik Chang (perky) Assigned to: Nobody/Anonymous (nobody) Summary: unichr(wide) error on ucs2 build Initial Comment: PyUnicode_FromOrdinal has code supporting UCS2 build already. But it is disabled by range validating routine. I think it should be either removed or modified to allow wide characters. (Please see the attachment.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926375&group_id=5470 From noreply at sourceforge.net Wed Mar 31 13:24:00 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 31 13:24:06 2004 Subject: [Patches] [ python-Patches-926860 ] Implementation for PEP 318 (Guido's version) Message-ID: Patches item #926860, was opened at 2004-03-31 13:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926860&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Nobody/Anonymous (nobody) Summary: Implementation for PEP 318 (Guido's version) Initial Comment: This patch (deco.diff) patches compile.c to recognize the following form of decorators: [list_of_expressions] def func(args): ... The list of expressions should contain at least one element and should not be a list comprehension, otherwise no special treatment is taken. (An empty list has no effect either way.) There's a simple test suite, Lib/test/test_decorators.py. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926860&group_id=5470 From noreply at sourceforge.net Wed Mar 31 13:52:09 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 31 13:52:16 2004 Subject: [Patches] [ python-Patches-926375 ] unichr(wide) error on ucs2 build Message-ID: Patches item #926375, was opened at 2004-03-31 02:51 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926375&group_id=5470 Category: Core (C code) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Hye-Shik Chang (perky) Assigned to: Nobody/Anonymous (nobody) Summary: unichr(wide) error on ucs2 build Initial Comment: PyUnicode_FromOrdinal has code supporting UCS2 build already. But it is disabled by range validating routine. I think it should be either removed or modified to allow wide characters. (Please see the attachment.) ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-31 20:52 Message: Logged In: YES user_id=21627 PEP 261 specifies that unichr() should raise a ValueError on narrow builds, so this check must stay the way it is. If you want to make it consistent, please remove the useless support for UTF-16 instead. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=926375&group_id=5470 From noreply at sourceforge.net Wed Mar 31 14:08:15 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 31 14:08:46 2004 Subject: [Patches] [ python-Patches-925932 ] struct.pack() on 64bit architectures Message-ID: Patches item #925932, was opened at 2004-03-30 12:55 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 Category: Tests Group: None Status: Open Resolution: None Priority: 5 Submitted By: Aleksander Piotrowski (pelotas) Assigned to: Nobody/Anonymous (nobody) Summary: struct.pack() on 64bit architectures Initial Comment: I'm running python on OpenBSD/sparc64 (SUN Ultra 10). On this machine struct.pack() gives me: >>> struct.pack('l', 1) '\x00\x00\x00\x00\x00\x00\x00\x01' >>> struct.pack('i', 1) '\x00\x00\x00\x01' On i386 box I have: >>> struct.pack('l', 1) '\x01\x00\x00\x00' >>> struct.pack('i', 1) '\x01\x00\x00\x00' Because of this, OpenBSD port uses attached patch. I guess that this is generic problem (not OpenBSD specific) and every 64bit platform would suffer. Am I right? ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-31 21:08 Message: Logged In: YES user_id=21627 I don't quite understand the current code: struct flock is on Darwin struct flock { off_t l_start; /* starting offset */ off_t l_len; /* len = 0 means until end of file */ pid_t l_pid; /* lock owner */ short l_type; /* lock type: read/write, etc. */ short l_whence; /* type of l_start */ }; and off_t is typedef int64_t quad_t; typedef quad_t off_t; /* file offset */ so it appears there is no padding at all. In any case, this needs to be changed in posixfile._posixfile_.flock accordingly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2004-03-30 17:57 Message: Logged In: YES user_id=31435 For a Unix-head reviewer: the patch is to test_fcntl.py, which uses a native struct.pack() to build a lockdata argument for fcntl. On the OP's box, it constructs stuff of the wrong size, due to "l" format codes producing 8-byte thingies. The patch changes the "l" codes to "i" on the OP's platform. I wonder whether we couldn't instead use "i" codes on all BSD- ish platforms -- on 32-bit boxes, "i" is synonymous with "l", while on 64-bit boxes it looks like "l" is wrong but "i" is correct. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=925932&group_id=5470 From noreply at sourceforge.net Wed Mar 31 14:11:24 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 31 15:02:33 2004 Subject: [Patches] [ python-Patches-924497 ] Updates to the Misc/RPM spec file. Message-ID: Patches item #924497, was opened at 2004-03-27 17:03 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=924497&group_id=5470 Category: Build Group: Python 2.3 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Sean Reifschneider (jafo) Assigned to: Nobody/Anonymous (nobody) Summary: Updates to the Misc/RPM spec file. Initial Comment: Because of continued problems with getting all the #!/usr/local/bin/python references out, I'm using a "find" to find the files that need to be fixed, instead of including a static list. Sean ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2004-03-31 21:11 Message: Logged In: YES user_id=21627 Thanks for the patch. Applied as python-2.3.spec 1.2.12.8 and 1.5 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=924497&group_id=5470 From duakmqvywpkzo at msn.com Wed Mar 31 05:25:33 2004 From: duakmqvywpkzo at msn.com (Raquel Moreland) Date: Wed Mar 31 18:17:29 2004 Subject: [Patches] degree without tests Message-ID: Diplomas from prestigious non-accredited universities NO required tests, classes, books, or interviews. Bachelors, Masters, MBA, and Doctorate (PhD) diplomas available in the field of your choice - that's right, you can become a Doctor and receive all the benefits and admiration that comes with it! No one is turned down. CALL TODAY ------->> 1 - 270-573-8493 (24 hours) <<----------- "I finally got my dream job that I knew I deserved. Thank you very much... " Fred Thompson, Kansas City, MO. "This got me a great job, that I never could have have without this program. Thank you. " Sharra Minnett , Chicago, IL From noreply at sourceforge.net Wed Mar 31 22:38:31 2004 From: noreply at sourceforge.net (SourceForge.net) Date: Wed Mar 31 22:38:34 2004 Subject: [Patches] [ python-Patches-927232 ] Option to propagate errors from pkgutil Message-ID: Patches item #927232, was opened at 2004-03-31 22:38 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=927232&group_id=5470 Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Jp Calderone (kuran) Assigned to: Nobody/Anonymous (nobody) Summary: Option to propagate errors from pkgutil Initial Comment: pkgutil currently catches IOErrors when opening files. Instead of letting the error be handled, it logs something to stdout. The attached patch adds an optional parameter to extend_path which allows this behavior to be changed to simply raise the exception. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=927232&group_id=5470