From noreply at sourceforge.net Fri Dec 1 21:05:35 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 01 Dec 2006 12:05:35 -0800 Subject: [Patches] [ python-Patches-1607087 ] popen() slow on AIX due to large FOPEN_MAX value Message-ID: Patches item #1607087, was opened at 2006-12-01 15:05 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607087&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: John L. Allen (johnlallen) Assigned to: Nobody/Anonymous (nobody) Summary: popen() slow on AIX due to large FOPEN_MAX value Initial Comment: Experimentation revealed that the os.popen[234]() family of methods was at least 10 times slower on AIX than it was on Solaris. It turns out that this is because of the _run_child method in popen2.py, which has this definition: def _run_child(self, cmd): if isinstance(cmd, basestring): cmd = ['/bin/sh', '-c', cmd] for i in xrange(3, MAXFD): try: os.close(i) except OSError: pass try: os.execvp(cmd[0], cmd) finally: os._exit(1) MAXFD is set as follows at the top of popen2.py: try: MAXFD = os.sysconf('SC_OPEN_MAX') except (AttributeError, ValueError): MAXFD = 256 On AIX, SC_OPEN_MAX is 32767, whereas on Solaris, it is not defined, and we get the default value of 256. So, on AIX the python code for loop is being used to close 32763 file descriptors, but only 253 on Solaris. The slowness of this python loop is the source of the problem. Several solutions are possible. AIX provides a much faster way to close all file descriptors above a given one using the fcntl F_CLOSEM option. In this case, it would be: fcntl(3, F_CLOSEM, 0). Other OSes, like Solaris and BSD, have the closefrom() function instead. I think ideally, we would want to have an os.closefrom() method defined in posixmodule.c, and always available on every OS, and have popen2.py call that instead of doing the loop. The closefrom() function would be defined something like this: -------------------- PyDoc_STRVAR(posix_closefrom__doc__, "closefrom(fd)\n\n\ Close all file descriptors greater than or equal to fd (for low level IO)."); static PyObject * posix_closefrom(PyObject *self, PyObject *args) { int fd, maxfd, res; if (!PyArg_ParseTuple(args, "i:closefrom", &fd)) return NULL; Py_BEGIN_ALLOW_THREADS #ifdef HAVE_CLOSEFROM res = closefrom(fd); #else #ifdef F_CLOSEM res = fcntl(3, F_CLOSEM, 0) #else #if defined( HAVE_SYSCONF ) && defined( _SC_OPEN_MAX ) # define PY_OPEN_MAX sysconf(_SC_OPEN_MAX) #else # ifdef FOPEN_MAX # define PY_OPEN_MAX FOPEN_MAX # else # ifdef OPEN_MAX # define PY_OPEN_MAX OPEN_MAX # else # ifdef _NFILE # define PY_OPEN_MAX _NFILE # else # define PY_OPEN_MAX 256 # endif # endif # endif #endif maxfd = PY_OPEN_MAX; while (fd < maxfd) { res = close(fd); fd++; } #endif #endif Py_END_ALLOW_THREADS if (res < 0) return posix_error(); Py_INCREF(Py_None); return Py_None; } --------------------- While this is probably (close to) the ideal solution (since it would benefit all OSes by avoiding the close loop if possible or if not, moving it to C code), adding os.closefrom() probably needs to be discussed further before being accepted by the Python community. Instead, I will provide a simpler patch that only benefits AIX. It adds the F_CLOSEM attribute to the fcntl class in fcntlmodule.c, if defined, and modifies popen2.py to check for it and use it if possible, instead of doing the close() loop. See the attached patch (against the 2.5 source). I don't believe that any documentation has to change. John Allen ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607087&group_id=5470 From noreply at sourceforge.net Fri Dec 1 22:07:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 01 Dec 2006 13:07:43 -0800 Subject: [Patches] [ python-Patches-1540874 ] broken shortcut keys Message-ID: Patches item #1540874, was opened at 2006-08-15 15:29 Message generated for change (Comment added) made by kbk You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1540874&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: Python 2.5 Status: Closed Resolution: Fixed Priority: 7 Private: No Submitted By: Jim Jewett (jimjjewett) Assigned to: Kurt B. Kaiser (kbk) Summary: broken shortcut keys Initial Comment: Idle menu entries can include a "_" indicating to use that character for a keyboard shortcut. Because Typing the key actually runs the shortcut (instead of just moving along the menu), there is no way to use the same shortcut on more than one entry -- listing it just misleads people into accidentally requesting the wrong command. Unfortunately, the default configuration does this for the File menu. It claims that Alt-f-p will save a copy as and print, but it actually just opens a path browser. This change (1) Uses the y key for save Cop_y (2) Stops pretending to have a key for print. I'll trust someone else's judgement on whether this is a bugfix (the advertised shortcuts don't work) or a feature (particularly the new binding for copy). ---------------------------------------------------------------------- >Comment By: Kurt B. Kaiser (kbk) Date: 2006-12-01 16:07 Message: Logged In: YES user_id=149084 Originator: NO If you were to work up a patch to display a Tk dialog, that would help until we can address printing properly. The current implementation is a quick cut by Guido. I suppose it goes to the default printer, but I've have to go read the code. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-21 09:49 Message: Logged In: YES user_id=1330769 Originator: NO Perhaps Ctrl+P is the standard keybinding for printing on Windows, but the windows standard is that the print command opens a print dialog. Therefore, is you invoked the command by mistake, you just hit "cancel" in the dialog. However this isn't the case in IDLE - IDLE just prints all of the text in the window. I've personally been bitten by this and several others, since we didn't know that Ctrl+P just sent the entire text to be printed. IDLE doesn't even notify you that you've sent the data for printing, so sometimes this is realized only after the entire thing has been printed. So if it's sticking to the standards we want, we should have IDLE pop up a "print..." dialog on Windows. (shouldn't be too hard with win32, is it possible without it?) ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2006-11-20 21:56 Message: Logged In: YES user_id=149084 Originator: NO Ctrl-P is the standard binding for Print on Windows. If you don't like it, assign it to some other key using your Options dialog. I don't want a patch, thanks anyway! ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2006-11-20 21:55 Message: Logged In: YES user_id=149084 Originator: NO Ctrl-P is the standard binding for Print on Windows. If you don't like it, assign it to some other key using your Options dialog. I don't want a patch, thanks anyway! ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-20 14:33 Message: Logged In: YES user_id=1330769 Originator: NO I actually prefer to not have a key for the "Print" item, since accidentally choosing it will print all of the code in the current window. On the same note, I think the default keybinding for the "Print" command shouldn't be Ctrl+P as it is now, for the same reason stated above. I prefer to have no key binding for it, since it isn't such a commonplace command, and invoking it accidentally is a major annoyance. However, currently IDLE assumes that every key in the keybindigs list must have a keybinding. If we don't want to change this, or if we insist on having a default keybinding for "Print", let's choose a combination that is less likely to be pressed accidentaly. (perhaps Ctrl+Alt+P) I could easily write a patch for this, just want to ask what you think should be done. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2006-08-16 17:48 Message: Logged In: YES user_id=149084 I chose 'T' for Print. Also modified the Shell menu hotkey, it's now 'L'. Rev 51329 ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-08-15 15:57 Message: Logged In: YES user_id=764593 Changing priority to 7 because I consider it a bug, but am reluctant to push GUI changes after release. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1540874&group_id=5470 From noreply at sourceforge.net Fri Dec 1 23:32:21 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 01 Dec 2006 14:32:21 -0800 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 14:32 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Sat Dec 2 21:53:34 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 02 Dec 2006 12:53:34 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Sun Dec 3 08:11:09 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 02 Dec 2006 23:11:09 -0800 Subject: [Patches] [ python-Patches-1607149 ] bug# 1607041: Condition.wait timeout fails on clock change Message-ID: Patches item #1607149, was opened at 2006-12-01 23:32 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: BC (hashstat) Assigned to: Nobody/Anonymous (nobody) Summary: bug# 1607041: Condition.wait timeout fails on clock change Initial Comment: This patch if for bug# 1607041. If the system clock is adjusted after Condition.wait is called with a timeout, the timeout does not expire as expected. This appears to be due to the threading.Condition class using the system clock to calculate the timeout expiration without taking system clock changes into account. No matter what timeout is used, setting the system clock ahead reduces or eliminates the wait while setting the system clock back increases the wait. So if the clock is set back one hour in the middle of a 1 microsecond wait (c.wait(1)), wait will return in an hour and 1 microsecond rather than after 1 microsecond. This patch modifies the Condition classes wait method to check for variations in the clock between calls to sleep and ajust for abnormalities. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 08:11 Message: Logged In: YES user_id=21627 Originator: NO It would be better if this was based on a monotonic clock if available on the system (such as the POSIX CLOCK_MONOTONIC argument to clock_gettime). "Guessing" jumps in the time is inherently unreliable; in your code, you won't notice changes that involve less than 10s. Notice that the same problem exists for Thread.join; it would be good if they were fixed together. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607149&group_id=5470 From noreply at sourceforge.net Sun Dec 3 12:24:30 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 03 Dec 2006 03:24:30 -0800 Subject: [Patches] [ python-Patches-1544279 ] Socket module is not thread-safe Message-ID: Patches item #1544279, was opened at 2006-08-22 01:24 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1544279&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None >Status: Closed >Resolution: Accepted Priority: 7 Private: No Submitted By: Maxim Sobolev (sobomax) Assigned to: Martin v. L?wis (loewis) Summary: Socket module is not thread-safe Initial Comment: The socket module make a great effort to be thread- safe, but misses one big point - it uses single per-instance buffer to hold resolved sockaddr_xxx structures. Therefore, on SMP system it is possible that several threads calling functions that perform address resolution in parallel will stomp on each other resulting in incorrect or invalid address to be used in each case. For example, two threads calling sendto() in parallel can result in packets to be sent to incorrect addresses - packets from thread one from time to time will be sent to address requested by thread two and vice versa. Another, smaller issue is that the call to getnameinfo () is not protected with netdb mutex on systems that don't have thread- safe resolver. P.S. This is very serious problem for us. For some reason my previous patch submission has been downgraded to bug report and forgotten completely, so that I am re-submitting. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 12:24 Message: Logged In: YES user_id=21627 Originator: NO Thanks for the patch. Committed as r52906 and r52907. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-08-22 01:42 Message: Logged In: YES user_id=21627 Sorry about forgetting your patch. Will look at it again for 2.5.1. P.S. The previous issue was #1467080. It wasn't "downgraded to bug report"; instead, you submitted it as a bug report to start with. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1544279&group_id=5470 From noreply at sourceforge.net Sun Dec 3 13:02:52 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 03 Dec 2006 04:02:52 -0800 Subject: [Patches] [ python-Patches-1371075 ] ConfigParser to accept a custom dict to allow ordering Message-ID: Patches item #1371075, was opened at 2005-12-01 18:39 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1371075&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Micah Elliott (mdelliot) Assigned to: Nobody/Anonymous (nobody) Summary: ConfigParser to accept a custom dict to allow ordering Initial Comment: In many cases it is preferrable for ConfigParser to maintain the order in which its items were added. This small change adds a `customdict` default argument to *ConfigParser's constructor to allow users to specify a custom dict. A useful example is:: from ConfigParser import ConfigParser ConfigParser(customdict=my_ordered_dict) One such `my_ordered_dict` could be the ordered dictionary "odict" . Ordered dictionaries have been a recent popular topic on comp.lang.python, and there are a few possibilities for implementation, so it seems best that ConfigParser just offer a default argument to support such customization. I can submit a documentation patch if this idea is accepted. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 13:02 Message: Logged In: YES user_id=21627 Originator: NO Thanks for the patch; I committed it with some modifications (e.g. renaming of the parameter to dict_type). Committed as r52908. Please include patches to the documentation and the test suite in the future. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1371075&group_id=5470 From noreply at sourceforge.net Sun Dec 3 13:32:55 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 03 Dec 2006 04:32:55 -0800 Subject: [Patches] [ python-Patches-1374063 ] Broader iterable support for xmlrpclib Message-ID: Patches item #1374063, was opened at 2005-12-06 05:14 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1374063&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Skip Montanaro (montanaro) >Assigned to: Skip Montanaro (montanaro) Summary: Broader iterable support for xmlrpclib Initial Comment: The XML-RPC spec supports a fairly limited set of datatypes. Most languages, Python included, contain many more types than XML-RPC. Some types, such as Python's complex number type, have no reasonable analog in XML-RPC. Others, such as unicode objects and array objects do. This patch allows anything that can be converted to a list but that is not otherwise supported directly by the xmlrpclib module already to be marshalled as an XML-RPC array if the allow_iter parameter to the ServerProxy constructor evaluated to true. This includes sets and arrays. Motivation... 1. Python already overloads the XML-RPC array type with both lists and tuples. This just extends that overloading to other currently unsupported Python types which can be converted to lists. Why should lists and tuples have all the fun? 2. Providing transparent conversion to XML-RPC arrays keeps calling code a bit cleaner. One of the attractions of XML-RPC is that the remote procedure call looks identical to a local call. This is especially true in Python because of /F's excellent little _Method class. Clearly as a programmer I could type: import array a = array.array('i', [1, 2,3]) ... from somemodule import somefunction print somefunction(list(a)) but that reveals details of the implementation of somefunction, namely that it can't handle arrays directly, even though in most respects arrays and lists are interchangeable. Attached is a patch for the xmlrpclib library that implements this feature, including minor doc changes and a new test case. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-03 13:32 Message: Logged In: YES user_id=21627 Originator: NO The patch is fine. Notice that it is currently out of date, due to inclusion of patch #1070046. This other patch added generic support for objects having an __dict__ attribute, marshaling them struct elements. This is in real conflict to this patch: an object both supporting a list conversion and having a __dict__ attribute could be marshaled either way. My proposal is to resolve this in favor of this patch: If an object has a list conversion, it's most likely meant to be a list. If you agree to this resolution, please update the code and the documentation, and apply this patch. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2005-12-06 05:23 Message: Logged In: YES user_id=44345 Oh, I forgot my original use case. I was constructing a list of musicians from a larger data structure and used a set to guarantee uniqueness during construction. I didn't really care about element ordering. I either had to convert the set to a list when calling the local function that made the RPC call, or modify the local function to always convert that arg to a list. Both alternatives seemed unattractive to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1374063&group_id=5470 From noreply at sourceforge.net Mon Dec 4 02:24:24 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 03 Dec 2006 17:24:24 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Mon Dec 4 06:29:10 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 03 Dec 2006 21:29:10 -0800 Subject: [Patches] [ python-Patches-1608267 ] Makefile fix Message-ID: Patches item #1608267, was opened at 2006-12-03 21:29 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hasan Diwan (hdiwan650) Assigned to: Nobody/Anonymous (nobody) Summary: Makefile fix Initial Comment: Create the DESTDIR as part of the make install process. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 From noreply at sourceforge.net Mon Dec 4 12:38:23 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 04 Dec 2006 03:38:23 -0800 Subject: [Patches] [ python-Patches-1565037 ] use LSB version information to detect a platform Message-ID: Patches item #1565037, was opened at 2006-09-25 14:06 Message generated for change (Comment added) made by doko You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1565037&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matthias Klose (doko) Assigned to: M.-A. Lemburg (lemburg) Summary: use LSB version information to detect a platform Initial Comment: reported at https://launchpad.net/bugs/54692 LSB conformant distributions should use lsb_version to get the platform information. ---------------------------------------------------------------------- >Comment By: Matthias Klose (doko) Date: 2006-12-04 12:38 Message: Logged In: YES user_id=60903 Originator: YES no, only the options for the lsb_release command are specified; the format of the file is unspecified. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2006-09-28 09:24 Message: Logged In: YES user_id=38388 Matthias, could you point me to a document defining the contents of that file ? Thanks. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-09-28 07:23 Message: Logged In: YES user_id=21627 MAL, can you please review this? If not, unassign. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1565037&group_id=5470 From noreply at sourceforge.net Mon Dec 4 17:52:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 04 Dec 2006 08:52:43 -0800 Subject: [Patches] [ python-Patches-1608579 ] Race condition in os.makedirs Message-ID: Patches item #1608579, was opened at 2006-12-04 11:52 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608579&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: James Bowes (jbowes) Assigned to: Nobody/Anonymous (nobody) Summary: Race condition in os.makedirs Initial Comment: os.makedirs first checks if the non-leaf part of the path exists, and then calls makedirs on this part of the path to create it. If the path is created in between the call to path.exists, and the recursive call to os.makedirs, then os.makedirs will raise an exception. The attached patch catches this exception, and then tests if the exception is caused by a the file/directory's existence. If it is, the exception is ignored. If not, it is rasied. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608579&group_id=5470 From noreply at sourceforge.net Mon Dec 4 21:53:39 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 04 Dec 2006 12:53:39 -0800 Subject: [Patches] [ python-Patches-1608758 ] Fix pickle doc typo "date", should be "data" Message-ID: Patches item #1608758, was opened at 2006-12-04 20:53 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608758&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Graham Horler (grahamh) Assigned to: Nobody/Anonymous (nobody) Summary: Fix pickle doc typo "date", should be "data" Initial Comment: Fix a typo in the pickle module documentation. It reads "date" where it should say "data". ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608758&group_id=5470 From noreply at sourceforge.net Mon Dec 4 23:06:35 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 04 Dec 2006 14:06:35 -0800 Subject: [Patches] [ python-Patches-1608805 ] Py_FileSystemDefaultEncoding can be non-canonical Message-ID: Patches item #1608805, was opened at 2006-12-04 17:06 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608805&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Stephan R.A. Deibel (sdeibel) Assigned to: Nobody/Anonymous (nobody) Summary: Py_FileSystemDefaultEncoding can be non-canonical Initial Comment: On Linux/Unix it is possible for Py_FileSystemDefaultEncoding to be set to a non-canonical encoding such as "UTF-8" instead of "utf-8". This happens when it is set from codeset in Py_InitializeEx() in pythonrun.c. This becomes a problem when this value is propagated through to PyUnicode_Decode() or PyUnicode_AsEncodedString() in unicodeobject.c. One possible such code path starts in os.listdir() via PyUnicode_FromEncodedObject()). In that case, the common case optimizations fail. I noticed this in a case where the PyCodec_Decode() used instead was failing. Normally I think this just amounts to broken optimization but given the likelihood of other such code being added in the future, I feel it's best to fix Py_FileSystemDefaultEncoding to always be a canonical form. One possible way to fix it is attached as a patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608805&group_id=5470 From noreply at sourceforge.net Tue Dec 5 06:42:12 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 04 Dec 2006 21:42:12 -0800 Subject: [Patches] [ python-Patches-1608758 ] Fix pickle doc typo "date", should be "data" Message-ID: Patches item #1608758, was opened at 2006-12-05 05:53 Message generated for change (Comment added) made by quiver You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608758&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.5 >Status: Closed Resolution: None Priority: 5 Private: No Submitted By: Graham Horler (grahamh) Assigned to: Nobody/Anonymous (nobody) Summary: Fix pickle doc typo "date", should be "data" Initial Comment: Fix a typo in the pickle module documentation. It reads "date" where it should say "data". ---------------------------------------------------------------------- >Comment By: George Yoshida (quiver) Date: 2006-12-05 14:42 Message: Logged In: YES user_id=671362 Originator: NO Applied in r52917(trunk) and r52918(25-maint). Thank you for your report. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608758&group_id=5470 From noreply at sourceforge.net Tue Dec 5 11:28:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 02:28:32 -0800 Subject: [Patches] [ python-Patches-1609108 ] Docstring support for ctypesgen Message-ID: Patches item #1609108, was opened at 2006-12-05 10:28 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609108&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Remahl (chmod007) Assigned to: Nobody/Anonymous (nobody) Summary: Docstring support for ctypesgen Initial Comment: ctypesgen should generate helpful docstrings for the functions it creates. This patch adds a "-d" switch to xml2py. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609108&group_id=5470 From noreply at sourceforge.net Tue Dec 5 16:16:38 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 07:16:38 -0800 Subject: [Patches] [ python-Patches-1609282 ] #1603424 subprocess.py wrongly claims 2.2 compatibility. Message-ID: Patches item #1609282, was opened at 2006-12-05 10:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609282&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Robert Carr (racarr) Assigned to: Nobody/Anonymous (nobody) Summary: #1603424 subprocess.py wrongly claims 2.2 compatibility. Initial Comment: Simple fix restoring 2.2 compatibility in subprocess.py. This makes more sense than a list comprehension or constructing sets in my opinion even ignoring the bug. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609282&group_id=5470 From noreply at sourceforge.net Tue Dec 5 17:43:47 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 08:43:47 -0800 Subject: [Patches] [ python-Patches-1609407 ] traceback on exit if syslog handler fails to initialize Message-ID: Patches item #1609407, was opened at 2006-12-05 11:43 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609407&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jeremy Katz (katzj) Assigned to: Nobody/Anonymous (nobody) Summary: traceback on exit if syslog handler fails to initialize Initial Comment: Syslog loggers can fail to initialize if the syslogd has crashed; this then leads to a traceback on exit as self.unixsocket isn't set. Attached patch changes around the order to avoid this. Related to http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=206474 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609407&group_id=5470 From noreply at sourceforge.net Tue Dec 5 18:54:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 09:54:22 -0800 Subject: [Patches] [ python-Patches-1609108 ] Docstring support for ctypesgen Message-ID: Patches item #1609108, was opened at 2006-12-05 11:28 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609108&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Invalid Priority: 5 Private: No Submitted By: David Remahl (chmod007) >Assigned to: Thomas Heller (theller) Summary: Docstring support for ctypesgen Initial Comment: ctypesgen should generate helpful docstrings for the functions it creates. This patch adds a "-d" switch to xml2py. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2006-12-05 18:54 Message: Logged In: YES user_id=11105 Originator: NO Since ctypeslib (which contains the code generator) is a thirdparty product I'm closing this. Bugs and patches for ctypes addon products should be submitted to the ctypes tracker at http://sourceforge.net/tracker/?group_id=71702. However, I'm happy to discuss this patch on the ctypes-users mailing list. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609108&group_id=5470 From noreply at sourceforge.net Tue Dec 5 23:55:03 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 14:55:03 -0800 Subject: [Patches] [ python-Patches-1608267 ] Makefile fix Message-ID: Patches item #1608267, was opened at 2006-12-04 05:29 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hasan Diwan (hdiwan650) Assigned to: Nobody/Anonymous (nobody) Summary: Makefile fix Initial Comment: Create the DESTDIR as part of the make install process. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 22:55 Message: Logged In: YES user_id=161998 Originator: NO Hello, * directory creation is usually done with install -d ; see numerous examples in the rest of the Makefile. * What does this fix? AFAIK, DESTDIR is already made, as part of directories like if test ! -d $(DESTDIR)$$i; then \ echo "Creating directory $$i"; \ $(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \ else true; \ ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 From noreply at sourceforge.net Wed Dec 6 00:40:54 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 15:40:54 -0800 Subject: [Patches] [ python-Patches-1507247 ] tarfile extraction does not honor umask Message-ID: Patches item #1507247, was opened at 2006-06-16 13:11 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Faik Uygur (faik) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile extraction does not honor umask Initial Comment: If the upperdirs in the member file's pathname does not exist. tarfile creates those paths with 0777 permission bits and does not honor umask. This patch uses umask to set the ti.mode of the created directory for later usage in chmod. --- tarfile.py (revision 46993) +++ tarfile.py (working copy) @@ -1560,7 +1560,9 @@ ti = TarInfo() ti.name = upperdirs ti.type = DIRTYPE - ti.mode = 0777 + umask = os.umask(0) + ti.mode = 0777 - umask + os.umask(umask) ti.mtime = tarinfo.mtime ti.uid = tarinfo.uid ti.gid = tarinfo.gid ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:40 Message: Logged In: YES user_id=161998 Originator: NO Hi, I can reproduce this problem on python 2.4 , and patch applies to python 2.5 too. Fix looks good to me. ---------------------------------------------------------------------- Comment By: Faik Uygur (faik) Date: 2006-08-18 10:44 Message: Logged In: YES user_id=1541018 Above patch is wrong. The correct one is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 From noreply at sourceforge.net Wed Dec 6 00:45:27 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 15:45:27 -0800 Subject: [Patches] [ python-Patches-1520879 ] make install change: Allow $DESTDIR to be relative Message-ID: Patches item #1520879, was opened at 2006-07-12 00:56 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1520879&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Douglas Greiman (dgreiman) Assigned to: Nobody/Anonymous (nobody) Summary: make install change: Allow $DESTDIR to be relative Initial Comment: This patch allows the $DESTDIR variable used in 'make install' to be a relative path. It also avoids constructing filenames starting with double slashes ('//') when DESTDIR is an absolute path. Tested on RedHat 9.0 Linux on x86. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:45 Message: Logged In: YES user_id=161998 Originator: NO Agreed. I have never seen $DESTDIR prefixed with / in makefiles, so this patch should go in. I don't see the point of making --prefix relative. you can do --prefix=`cd ../relative ; pwd` if you need relative. Putting relative paths in makefiles is asking for trouble, as the Makefile will call other makefiles in subdirectories, which would need to pass extra ../ overrides of prefix. ---------------------------------------------------------------------- Comment By: Douglas Greiman (dgreiman) Date: 2006-08-01 00:03 Message: Logged In: YES user_id=1553997 That seems reasonable, however I believe it should be a separate change since codewise the two changes are unrelated. ---------------------------------------------------------------------- Comment By: Chad Whitacre (whit537) Date: 2006-07-31 22:13 Message: Logged In: YES user_id=340931 I'd also like to see the argument to 'configure --prefix=' also be relative. Could this patch be expanded in that direction? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1520879&group_id=5470 From noreply at sourceforge.net Wed Dec 6 00:51:59 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 15:51:59 -0800 Subject: [Patches] [ python-Patches-1339673 ] cross compile and mingw support Message-ID: Patches item #1339673, was opened at 2005-10-27 17:18 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1339673&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jan Nieuwenhuizen (janneke-sf) Assigned to: Nobody/Anonymous (nobody) Summary: cross compile and mingw support Initial Comment: The cross compile patch is similar to what I did for Guile and has still some bits from Daniel Goertzen's https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1006238&group_id=5470 The mingw patch is based on Matthias Kramm's https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1053879&group_id=5470 but adds autoconf tests and does have his C modifications. -- Jan Nieuwenhuizen | GNU LilyPond - The music typesetter http://www.xs4all.nl/~jantien | http://www.lilypond.org ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:51 Message: Logged In: YES user_id=161998 Originator: NO superseded by #1597850 ---------------------------------------------------------------------- Comment By: Richard Tew (rmt38) Date: 2006-01-06 05:46 Message: Logged In: YES user_id=1417949 > Why do you think that? You will find that no autotooled > package will build or install in directories with spaces Perhaps I was not clear enough. The problem is not that I am building in those directories, it is that the patch sets the PATH environment variable as a prefix to the compilation command (in the same line). And on Windows, it is not uncommon for directories in Program Files and other directories with spaces in their names to be in the path. This breaks the correctness of the command line and the execution of it. Take another look at the erroneous command: PATH=/usr/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/WINDOWS/s ystem32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/Program Files/ATI Technologies/ATI Control Panel:/c/PROGRA~1/COMMON~1/SONICS~1/:/c/Program Files/Common Files/Adobe/AGL:/usr/bin:/c/devkitpro/devkitarm/bin:. cc - c -I -I./Include -o Parser/acceler.x Parser/acceler.c And the error: /bin/sh: Files/ATI: No such file or directory make: *** [Parser/acceler.x] Error 127 Hope this helps. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2006-01-06 00:31 Message: Logged In: YES user_id=1368960 > Looks like the patch needs to handle paths with spaces in, > in the PATH. Why do you think that? You will find that no autotooled package will build or install in directories with spaces. If this would be possible at all, it would be a separate feature. I do not see a dependency with cross compilation or mingw building. ---------------------------------------------------------------------- Comment By: Richard Tew (rmt38) Date: 2006-01-05 21:20 Message: Logged In: YES user_id=1417949 Hi, I patched Python 2.4.2 source code download with this (can't get CVS as bandwidth is limited). context-cross.patch fails on one chunk of Makefile.pre.in, although hand application fixed it. Otherwise, context- mingw-2.patch applied cleanly. With both applied, I did the following: export BASECFLAGS="-mcpu=arm9tdmi -mcpu=arm9tdmi -ffast- math -mthumb -mthumb-interwork -DARM9 -D__NDS__" export CFLAGS="-mcpu=arm9tdmi -mcpu=arm9tdmi -ffast-math - mthumb -mthumb-interwork -DARM9 -D__NDS__" export LDFLAGS="-specs=ds_arm9.specs -g -mthumb -mthumb- interwork -Wl,-Map,libpython2.4.map" sh configure --host=arm-elf This is what eventually happened: PATH=/usr/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/WINDOWS/s ystem32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/Program Files/ATI Technologies/ATI Control Panel:/c/PROGRA~1/COMMON~1/SONICS~1/:/c/Program Files/Common Files/Adobe/AGL:/usr/bin:/c/devkitpro/devkitarm/bin:. cc - c -I -I./Include -o Parser/acceler.x Parser/acceler.c /bin/sh: Files/ATI: No such file or directory make: *** [Parser/acceler.x] Error 127 Looks like the patch needs to handle paths with spaces in, in the PATH. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2005-11-04 13:59 Message: Logged In: YES user_id=1368960 The second mingw patch actually produces a working python, fixing * loadable dll modules * argv0 relocation * posixflavour nt -- Jan. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2005-10-27 17:20 Message: Logged In: YES user_id=1368960 .... and does NOT have his C modifications ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1339673&group_id=5470 From noreply at sourceforge.net Wed Dec 6 00:52:14 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 15:52:14 -0800 Subject: [Patches] [ python-Patches-1339673 ] cross compile and mingw support Message-ID: Patches item #1339673, was opened at 2005-10-27 17:18 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1339673&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jan Nieuwenhuizen (janneke-sf) Assigned to: Nobody/Anonymous (nobody) Summary: cross compile and mingw support Initial Comment: The cross compile patch is similar to what I did for Guile and has still some bits from Daniel Goertzen's https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1006238&group_id=5470 The mingw patch is based on Matthias Kramm's https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1053879&group_id=5470 but adds autoconf tests and does have his C modifications. -- Jan Nieuwenhuizen | GNU LilyPond - The music typesetter http://www.xs4all.nl/~jantien | http://www.lilypond.org ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:52 Message: Logged In: YES user_id=161998 Originator: NO superseded by #1597850 ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:51 Message: Logged In: YES user_id=161998 Originator: NO superseded by #1597850 ---------------------------------------------------------------------- Comment By: Richard Tew (rmt38) Date: 2006-01-06 05:46 Message: Logged In: YES user_id=1417949 > Why do you think that? You will find that no autotooled > package will build or install in directories with spaces Perhaps I was not clear enough. The problem is not that I am building in those directories, it is that the patch sets the PATH environment variable as a prefix to the compilation command (in the same line). And on Windows, it is not uncommon for directories in Program Files and other directories with spaces in their names to be in the path. This breaks the correctness of the command line and the execution of it. Take another look at the erroneous command: PATH=/usr/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/WINDOWS/s ystem32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/Program Files/ATI Technologies/ATI Control Panel:/c/PROGRA~1/COMMON~1/SONICS~1/:/c/Program Files/Common Files/Adobe/AGL:/usr/bin:/c/devkitpro/devkitarm/bin:. cc - c -I -I./Include -o Parser/acceler.x Parser/acceler.c And the error: /bin/sh: Files/ATI: No such file or directory make: *** [Parser/acceler.x] Error 127 Hope this helps. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2006-01-06 00:31 Message: Logged In: YES user_id=1368960 > Looks like the patch needs to handle paths with spaces in, > in the PATH. Why do you think that? You will find that no autotooled package will build or install in directories with spaces. If this would be possible at all, it would be a separate feature. I do not see a dependency with cross compilation or mingw building. ---------------------------------------------------------------------- Comment By: Richard Tew (rmt38) Date: 2006-01-05 21:20 Message: Logged In: YES user_id=1417949 Hi, I patched Python 2.4.2 source code download with this (can't get CVS as bandwidth is limited). context-cross.patch fails on one chunk of Makefile.pre.in, although hand application fixed it. Otherwise, context- mingw-2.patch applied cleanly. With both applied, I did the following: export BASECFLAGS="-mcpu=arm9tdmi -mcpu=arm9tdmi -ffast- math -mthumb -mthumb-interwork -DARM9 -D__NDS__" export CFLAGS="-mcpu=arm9tdmi -mcpu=arm9tdmi -ffast-math - mthumb -mthumb-interwork -DARM9 -D__NDS__" export LDFLAGS="-specs=ds_arm9.specs -g -mthumb -mthumb- interwork -Wl,-Map,libpython2.4.map" sh configure --host=arm-elf This is what eventually happened: PATH=/usr/bin:.:/usr/local/bin:/mingw/bin:/bin:/c/WINDOWS/s ystem32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:/c/Program Files/ATI Technologies/ATI Control Panel:/c/PROGRA~1/COMMON~1/SONICS~1/:/c/Program Files/Common Files/Adobe/AGL:/usr/bin:/c/devkitpro/devkitarm/bin:. cc - c -I -I./Include -o Parser/acceler.x Parser/acceler.c /bin/sh: Files/ATI: No such file or directory make: *** [Parser/acceler.x] Error 127 Looks like the patch needs to handle paths with spaces in, in the PATH. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2005-11-04 13:59 Message: Logged In: YES user_id=1368960 The second mingw patch actually produces a working python, fixing * loadable dll modules * argv0 relocation * posixflavour nt -- Jan. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2005-10-27 17:20 Message: Logged In: YES user_id=1368960 .... and does NOT have his C modifications ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1339673&group_id=5470 From noreply at sourceforge.net Wed Dec 6 00:58:50 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 05 Dec 2006 15:58:50 -0800 Subject: [Patches] [ python-Patches-1608579 ] Race condition in os.makedirs Message-ID: Patches item #1608579, was opened at 2006-12-04 16:52 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608579&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: James Bowes (jbowes) Assigned to: Nobody/Anonymous (nobody) Summary: Race condition in os.makedirs Initial Comment: os.makedirs first checks if the non-leaf part of the path exists, and then calls makedirs on this part of the path to create it. If the path is created in between the call to path.exists, and the recursive call to os.makedirs, then os.makedirs will raise an exception. The attached patch catches this exception, and then tests if the exception is caused by a the file/directory's existence. If it is, the exception is ignored. If not, it is rasied. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:58 Message: Logged In: YES user_id=161998 Originator: NO looks good to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608579&group_id=5470 From noreply at sourceforge.net Wed Dec 6 12:30:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 03:30:43 -0800 Subject: [Patches] [ python-Patches-1555570 ] email parser incorrectly breaks headers with a CRLF at 8192 Message-ID: Patches item #1555570, was opened at 2006-09-09 23:41 Message generated for change (Comment added) made by graham_king You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1555570&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Meyer (anadelonbrin) Assigned to: Barry A. Warsaw (bwarsaw) Summary: email parser incorrectly breaks headers with a CRLF at 8192 Initial Comment: If a message has a CRLF as part of a header that starts at 8191, the parser will incorrectly consider the headers to finish at 8191 and the body to start at 8193, which leaves headers in the body of the message. This problem occurs because the parser reads 8192 characters at a time. If 8192 is a '\r' and 8193 a '\n', then when the second block is parsed, it will appear to be a blank line (i.e. header separator). The simplest fix for this is to just read an extra character if the last one read is a '\r'. This appears to work fine without breaking anything, although I suppose that an alternative would be to change the FeedParser to check whether the '\n' belonged with the previous data. A patch and test are attached, against SVN of 10/Sept/06. ---------------------------------------------------------------------- Comment By: Graham King (graham_king) Date: 2006-12-06 11:30 Message: Logged In: YES user_id=1237146 Originator: NO I had exactly the same problem parsing a POST encoded as 'multipart/form-data' with lots of parts. The patch provided fixes it. I'm on ActivePython 2.4.3 so I applied the patch by hand. +1 for integrating this patch into Python. Thanks, Graham King. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1555570&group_id=5470 From noreply at sourceforge.net Wed Dec 6 15:24:09 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 06:24:09 -0800 Subject: [Patches] [ python-Patches-1608267 ] Makefile fix Message-ID: Patches item #1608267, was opened at 2006-12-03 21:29 Message generated for change (Comment added) made by hdiwan650 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hasan Diwan (hdiwan650) Assigned to: Nobody/Anonymous (nobody) Summary: Makefile fix Initial Comment: Create the DESTDIR as part of the make install process. ---------------------------------------------------------------------- >Comment By: Hasan Diwan (hdiwan650) Date: 2006-12-06 06:24 Message: Logged In: YES user_id=1185570 Originator: YES Ok, I'll change the patch to use install as opposed to mkdir in a few days. I was hitting a problem in installing python, which I have since solved by using this patch. So, I decided to submit it. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 14:55 Message: Logged In: YES user_id=161998 Originator: NO Hello, * directory creation is usually done with install -d ; see numerous examples in the rest of the Makefile. * What does this fix? AFAIK, DESTDIR is already made, as part of directories like if test ! -d $(DESTDIR)$$i; then \ echo "Creating directory $$i"; \ $(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \ else true; \ ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 From noreply at sourceforge.net Wed Dec 6 15:25:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 06:25:37 -0800 Subject: [Patches] [ python-Patches-1608267 ] Makefile fix Message-ID: Patches item #1608267, was opened at 2006-12-04 05:29 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Hasan Diwan (hdiwan650) Assigned to: Nobody/Anonymous (nobody) Summary: Makefile fix Initial Comment: Create the DESTDIR as part of the make install process. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-06 14:25 Message: Logged In: YES user_id=161998 Originator: NO can you be more specific about the problem? the install -d should come just before the command attempting to copy something into that dir. ---------------------------------------------------------------------- Comment By: Hasan Diwan (hdiwan650) Date: 2006-12-06 14:24 Message: Logged In: YES user_id=1185570 Originator: YES Ok, I'll change the patch to use install as opposed to mkdir in a few days. I was hitting a problem in installing python, which I have since solved by using this patch. So, I decided to submit it. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 22:55 Message: Logged In: YES user_id=161998 Originator: NO Hello, * directory creation is usually done with install -d ; see numerous examples in the rest of the Makefile. * What does this fix? AFAIK, DESTDIR is already made, as part of directories like if test ! -d $(DESTDIR)$$i; then \ echo "Creating directory $$i"; \ $(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \ else true; \ ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608267&group_id=5470 From noreply at sourceforge.net Wed Dec 6 18:54:11 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 09:54:11 -0800 Subject: [Patches] [ python-Patches-1555570 ] email parser incorrectly breaks headers with a CRLF at 8192 Message-ID: Patches item #1555570, was opened at 2006-09-09 18:41 Message generated for change (Comment added) made by jdunck You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1555570&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Meyer (anadelonbrin) Assigned to: Barry A. Warsaw (bwarsaw) Summary: email parser incorrectly breaks headers with a CRLF at 8192 Initial Comment: If a message has a CRLF as part of a header that starts at 8191, the parser will incorrectly consider the headers to finish at 8191 and the body to start at 8193, which leaves headers in the body of the message. This problem occurs because the parser reads 8192 characters at a time. If 8192 is a '\r' and 8193 a '\n', then when the second block is parsed, it will appear to be a blank line (i.e. header separator). The simplest fix for this is to just read an extra character if the last one read is a '\r'. This appears to work fine without breaking anything, although I suppose that an alternative would be to change the FeedParser to check whether the '\n' belonged with the previous data. A patch and test are attached, against SVN of 10/Sept/06. ---------------------------------------------------------------------- Comment By: Jeremy Dunck (jdunck) Date: 2006-12-06 11:54 Message: Logged In: YES user_id=516425 Originator: NO We're hitting this bug as well. It affects Django, which handles media upload over HTTP by parsing multipart with email.message_from_string. pegasusnews.com, FWIW. ---------------------------------------------------------------------- Comment By: Graham King (graham_king) Date: 2006-12-06 05:30 Message: Logged In: YES user_id=1237146 Originator: NO I had exactly the same problem parsing a POST encoded as 'multipart/form-data' with lots of parts. The patch provided fixes it. I'm on ActivePython 2.4.3 so I applied the patch by hand. +1 for integrating this patch into Python. Thanks, Graham King. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1555570&group_id=5470 From noreply at sourceforge.net Wed Dec 6 21:06:24 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 12:06:24 -0800 Subject: [Patches] [ python-Patches-1597850 ] Cross compiling patches for MINGW Message-ID: Patches item #1597850, was opened at 2006-11-16 17:57 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Han-Wen Nienhuys (hanwen) Assigned to: Nobody/Anonymous (nobody) Summary: Cross compiling patches for MINGW Initial Comment: Hello, attached tarbal is a patch bomb of 32 patches against python 2.5, that we lilypond developers use for crosscompiling python. The patches were originally written by Jan Nieuwenhuizen, my codeveloper. These patches have been tested with Linux/x86, linux/x64 and macos 10.3 as build host and linux-{ppc,x86,x86_64}, freebsd, mingw as target platform. All packages at lilypond.org/install/ except for darwin contain the x-compiled python. Each patch is prefixed with a small comment, but for reference, I include a snippet from the readme. It would be nice if at least some of the patches were included. In particular, I think that X-compiling is a common request, so it warrants inclusion. Basically, what we do is override autoconf and Makefile settings through setting enviroment variables. **README section** Cross Compiling --------------- Python can be cross compiled by supplying different --build and --host parameters to configure. Python is compiled on the "build" system and executed on the "host" system. Cross compiling python requires a native Python on the build host, and a natively compiled tool `Pgen'. Before cross compiling, Python must first be compiled and installed on the build host. The configure script will use `cc' and `python', or environment variables CC_FOR_BUILD or PYTHON_FOR_BUILD, eg: CC_FOR_BUILD=gcc-3.3 \ PYTHON_FOR_BUILD=python2.4 \ .../configure --build=i686-linux --host=i586-mingw32 Cross compiling has been tested under linux, mileage may vary for other platforms. A few reminders on using configure to cross compile: - Cross compile tools must be in PATH, - Cross compile tools must be prefixed with the host type (ie i586-mingw32-gcc, i586-mingw32-ranlib, ...), - CC, CXX, AR, and RANLIB must be undefined when running configure, they will be auto-detected. If you need a cross compiler, Debian ships several several (eg: avr, m68hc1x, mingw32), while dpkg-cross easily creates others. Otherwise, check out Dan Kegel's crosstool: http://www.kegel.com/crosstool . ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-06 21:06 Message: Logged In: YES user_id=21627 Originator: NO I'll add my comments as I go through the patches. cab1e7d1e54d14a8aab52f0c3b3073c93f75d4fc: - why is there now a mingw32msvc2 platform? If the target is mingw (rather than Cygwin), I'd expect that the target is just Win32/Windows, and that all symbolic constants provided be usable across all Win32 Pythons. - why is h2py run for /usr/include/netinet/in.h? Shouldn't it operate on a target header file? - please include any plat-* files that you generate in the patch. - why do you need dl_nt.c in Modules? Please make it use the one from PC (consider updating the comment about calling initall) b52dbbbbc3adece61496b161d8c22599caae2311 - please combine all patches adding support for __MINGW32__ into a single one. Why is anything needed here at all? I thought Python compiles already with mingw32 (on Windows)? - what is the exclusion of freezing for? 059af829d362b10bb5921367c93a56dbb51ef31b - Why are you taking timeval from winsock2.h? It should come from sys/time.h, and does in my copy of Debian mingw32-runtime. 6a742fb15b28564f9a1bc916c76a28dc672a9b2c - Why are these changes needed? It's Windows, and that is already supported. a838b4780998ef98ae4880c3916274d45b661c82 - Why doesn't that already work on Windows+cygwin+mingw32? f452fe4b95085d8c1ba838bf302a6a48df3c1d31 - I think this should target msvcr71.dll, not msvcrt.dll Please also combine the cross-compilation patches into a single one. - there is no need to provide pyconfig.h.in changes; I'll regenerate that, anyway. 9c022e407c366c9f175e9168542ccc76eae9e3f0 - please integrate those into the large AC_CHECK_FUNCS that already exists 540684d696df6057ee2c9c4e13e33fe450605ffa - Why are you stripping -Wl? 64f5018e975419b2d37c39f457c8732def3288df - Try getting SO from the Makefile, not from the environment (I assume this is also meant to support true distutils packages some day). 7a4e50fb1cf5ff3481aaf7515a784621cbbdac6c - again: what is the "mingw" platform? 7d3a45788a0d83608d10e5c0a34f08b426d62e92 - is this really necessary? I suggest to drop it 23a2dd14933a2aee69f7cdc9f838e4b9c26c1eea - don't include bits/time.h; it's not meant for direct inclusion 6689ca9dea07afbe8a77b7787a5c4e1642f803a1 - what's a .x file? ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-25 16:12 Message: Logged In: YES user_id=161998 Originator: YES I've sent the agreement by snailmail. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2006-11-17 20:57 Message: Logged In: YES user_id=1368960 Originator: NO I do not mind either. I've just signed and faxed contrib-form.html. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 01:33 Message: Logged In: YES user_id=161998 Originator: YES note that not all of the patch needs to go in its current form. In particular, setup.py should be much more clever to look into build-root for finding libs and include files. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 01:32 Message: Logged In: YES user_id=161998 Originator: YES I don't mind, and I expect Jan won't have a problem either. What's the procedure: do we send the disclaimer first, or do you do the review, or does everything happen in parallel? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-16 22:47 Message: Logged In: YES user_id=21627 Originator: NO Would you and Jan Nieuwenhuizen be willing to sign the contributor agreement, at http://www.python.org/psf/contrib.html I haven't reviewed the patch yet; if they can be integrated, that will only happen in the trunk (i.e. not for 2.5.x). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 From noreply at sourceforge.net Wed Dec 6 21:12:23 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 12:12:23 -0800 Subject: [Patches] [ python-Patches-1597850 ] Cross compiling patches for MINGW Message-ID: Patches item #1597850, was opened at 2006-11-16 17:57 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Han-Wen Nienhuys (hanwen) Assigned to: Nobody/Anonymous (nobody) Summary: Cross compiling patches for MINGW Initial Comment: Hello, attached tarbal is a patch bomb of 32 patches against python 2.5, that we lilypond developers use for crosscompiling python. The patches were originally written by Jan Nieuwenhuizen, my codeveloper. These patches have been tested with Linux/x86, linux/x64 and macos 10.3 as build host and linux-{ppc,x86,x86_64}, freebsd, mingw as target platform. All packages at lilypond.org/install/ except for darwin contain the x-compiled python. Each patch is prefixed with a small comment, but for reference, I include a snippet from the readme. It would be nice if at least some of the patches were included. In particular, I think that X-compiling is a common request, so it warrants inclusion. Basically, what we do is override autoconf and Makefile settings through setting enviroment variables. **README section** Cross Compiling --------------- Python can be cross compiled by supplying different --build and --host parameters to configure. Python is compiled on the "build" system and executed on the "host" system. Cross compiling python requires a native Python on the build host, and a natively compiled tool `Pgen'. Before cross compiling, Python must first be compiled and installed on the build host. The configure script will use `cc' and `python', or environment variables CC_FOR_BUILD or PYTHON_FOR_BUILD, eg: CC_FOR_BUILD=gcc-3.3 \ PYTHON_FOR_BUILD=python2.4 \ .../configure --build=i686-linux --host=i586-mingw32 Cross compiling has been tested under linux, mileage may vary for other platforms. A few reminders on using configure to cross compile: - Cross compile tools must be in PATH, - Cross compile tools must be prefixed with the host type (ie i586-mingw32-gcc, i586-mingw32-ranlib, ...), - CC, CXX, AR, and RANLIB must be undefined when running configure, they will be auto-detected. If you need a cross compiler, Debian ships several several (eg: avr, m68hc1x, mingw32), while dpkg-cross easily creates others. Otherwise, check out Dan Kegel's crosstool: http://www.kegel.com/crosstool . ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-06 21:12 Message: Logged In: YES user_id=21627 Originator: NO One more note: it would be best if the patches were against the subversion trunk. They won't be included in the 2.5 maintenance branch (as they are a new feature), so they need to be ported to the trunk, anyway. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-06 21:06 Message: Logged In: YES user_id=21627 Originator: NO I'll add my comments as I go through the patches. cab1e7d1e54d14a8aab52f0c3b3073c93f75d4fc: - why is there now a mingw32msvc2 platform? If the target is mingw (rather than Cygwin), I'd expect that the target is just Win32/Windows, and that all symbolic constants provided be usable across all Win32 Pythons. - why is h2py run for /usr/include/netinet/in.h? Shouldn't it operate on a target header file? - please include any plat-* files that you generate in the patch. - why do you need dl_nt.c in Modules? Please make it use the one from PC (consider updating the comment about calling initall) b52dbbbbc3adece61496b161d8c22599caae2311 - please combine all patches adding support for __MINGW32__ into a single one. Why is anything needed here at all? I thought Python compiles already with mingw32 (on Windows)? - what is the exclusion of freezing for? 059af829d362b10bb5921367c93a56dbb51ef31b - Why are you taking timeval from winsock2.h? It should come from sys/time.h, and does in my copy of Debian mingw32-runtime. 6a742fb15b28564f9a1bc916c76a28dc672a9b2c - Why are these changes needed? It's Windows, and that is already supported. a838b4780998ef98ae4880c3916274d45b661c82 - Why doesn't that already work on Windows+cygwin+mingw32? f452fe4b95085d8c1ba838bf302a6a48df3c1d31 - I think this should target msvcr71.dll, not msvcrt.dll Please also combine the cross-compilation patches into a single one. - there is no need to provide pyconfig.h.in changes; I'll regenerate that, anyway. 9c022e407c366c9f175e9168542ccc76eae9e3f0 - please integrate those into the large AC_CHECK_FUNCS that already exists 540684d696df6057ee2c9c4e13e33fe450605ffa - Why are you stripping -Wl? 64f5018e975419b2d37c39f457c8732def3288df - Try getting SO from the Makefile, not from the environment (I assume this is also meant to support true distutils packages some day). 7a4e50fb1cf5ff3481aaf7515a784621cbbdac6c - again: what is the "mingw" platform? 7d3a45788a0d83608d10e5c0a34f08b426d62e92 - is this really necessary? I suggest to drop it 23a2dd14933a2aee69f7cdc9f838e4b9c26c1eea - don't include bits/time.h; it's not meant for direct inclusion 6689ca9dea07afbe8a77b7787a5c4e1642f803a1 - what's a .x file? ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-25 16:12 Message: Logged In: YES user_id=161998 Originator: YES I've sent the agreement by snailmail. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2006-11-17 20:57 Message: Logged In: YES user_id=1368960 Originator: NO I do not mind either. I've just signed and faxed contrib-form.html. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 01:33 Message: Logged In: YES user_id=161998 Originator: YES note that not all of the patch needs to go in its current form. In particular, setup.py should be much more clever to look into build-root for finding libs and include files. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 01:32 Message: Logged In: YES user_id=161998 Originator: YES I don't mind, and I expect Jan won't have a problem either. What's the procedure: do we send the disclaimer first, or do you do the review, or does everything happen in parallel? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-16 22:47 Message: Logged In: YES user_id=21627 Originator: NO Would you and Jan Nieuwenhuizen be willing to sign the contributor agreement, at http://www.python.org/psf/contrib.html I haven't reviewed the patch yet; if they can be integrated, that will only happen in the trunk (i.e. not for 2.5.x). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 From noreply at sourceforge.net Wed Dec 6 23:04:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 14:04:32 -0800 Subject: [Patches] [ python-Patches-1610437 ] tarfile.py: Fix for bug #1609958 Message-ID: Patches item #1610437, was opened at 2006-12-06 23:04 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610437&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py: Fix for bug #1609958 Initial Comment: My last patch to tarfile.py (#1583880) introduced a bug that leads to miscalculation of header checksums for members that need the GNU longname extensions. Sadly, this bug slipped through the regression tests. The attached patch fixes the problem and extends the existing test case. Additionally, it fixes a minor issue in tarfile.py which I just came across regarding the "././@LongLink" placeholder. Both 2.6 and 2.5 need this bugfix. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610437&group_id=5470 From noreply at sourceforge.net Wed Dec 6 23:21:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 14:21:32 -0800 Subject: [Patches] [ python-Patches-1610437 ] tarfile.py: Fix for bug #1609958 Message-ID: Patches item #1610437, was opened at 2006-12-06 22:04 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610437&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py: Fix for bug #1609958 Initial Comment: My last patch to tarfile.py (#1583880) introduced a bug that leads to miscalculation of header checksums for members that need the GNU longname extensions. Sadly, this bug slipped through the regression tests. The attached patch fixes the problem and extends the existing test case. Additionally, it fixes a minor issue in tarfile.py which I just came across regarding the "././@LongLink" placeholder. Both 2.6 and 2.5 need this bugfix. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-06 22:21 Message: Logged In: YES user_id=849994 Originator: NO Committed as rev. 52938, 52939 (2.5). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610437&group_id=5470 From noreply at sourceforge.net Thu Dec 7 06:37:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 06 Dec 2006 21:37:43 -0800 Subject: [Patches] [ python-Patches-1610575 ] C99 _Bool support for struct Message-ID: Patches item #1610575, was opened at 2006-12-07 05:37 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610575&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Remahl (chmod007) Assigned to: Nobody/Anonymous (nobody) Summary: C99 _Bool support for struct Initial Comment: C99 adds the fundamental _Bool integer type (fundamental in the sense that it is not equivalent to or a composite of any other C type). Its size can vary from platform to platform; the only restriction imposed by the C standard is that it must be able to hold the values 0 or 1. Typically, sizeof _Bool is 1 or 4. A struct module user trying to parse a native C structure that contains a _Bool member faces a problem: struct does not have a format character for _Bool. One is forced to hardcode a size for bool (use a char or an int instead). This patch adds support for a new format character, 't', representing the fundamental type _Bool. It is handled sementically as representing pure booleans -- when packing a structure the truth value of the argument to be packed is used and when unpacking either True or False is always returned. For platforms that don't support _Bool, as well as in non-native mode, 't' packs as a single byte. Test cases are included, as well as a small change to the struct documentation. The patch modifies configure.in to check for _Bool support, and the patch includes the autogenerated configure and pyconfig.h.in files as well. I have tested the module on Mac OS X x86 (uses 1 byte for _Bool) and Mac OS X ppc (uses 4 bytes for _Bool). Ran regression suite. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610575&group_id=5470 From noreply at sourceforge.net Thu Dec 7 14:29:00 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 07 Dec 2006 05:29:00 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 14:29 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) Assigned to: Nobody/Anonymous (nobody) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Thu Dec 7 20:15:17 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 07 Dec 2006 11:15:17 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 14:29 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) >Assigned to: Thomas Heller (theller) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-07 20:15 Message: Logged In: YES user_id=21627 Originator: NO Thomas, can you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Thu Dec 7 21:09:39 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 07 Dec 2006 12:09:39 -0800 Subject: [Patches] [ python-Patches-1610575 ] C99 _Bool support for struct Message-ID: Patches item #1610575, was opened at 2006-12-07 06:37 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610575&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Remahl (chmod007) Assigned to: Nobody/Anonymous (nobody) Summary: C99 _Bool support for struct Initial Comment: C99 adds the fundamental _Bool integer type (fundamental in the sense that it is not equivalent to or a composite of any other C type). Its size can vary from platform to platform; the only restriction imposed by the C standard is that it must be able to hold the values 0 or 1. Typically, sizeof _Bool is 1 or 4. A struct module user trying to parse a native C structure that contains a _Bool member faces a problem: struct does not have a format character for _Bool. One is forced to hardcode a size for bool (use a char or an int instead). This patch adds support for a new format character, 't', representing the fundamental type _Bool. It is handled sementically as representing pure booleans -- when packing a structure the truth value of the argument to be packed is used and when unpacking either True or False is always returned. For platforms that don't support _Bool, as well as in non-native mode, 't' packs as a single byte. Test cases are included, as well as a small change to the struct documentation. The patch modifies configure.in to check for _Bool support, and the patch includes the autogenerated configure and pyconfig.h.in files as well. I have tested the module on Mac OS X x86 (uses 1 byte for _Bool) and Mac OS X ppc (uses 4 bytes for _Bool). Ran regression suite. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2006-12-07 21:09 Message: Logged In: YES user_id=11105 Originator: NO The patch is not complete or not correct. Either: - the part of that patch that changes Modules/_ctypes/_ctypes.c should be omitted because it does not contain a ctypes _Bool type - or complete support for a ctypes _Bool type (how would that be called? ctypes.c99_bool?) should be added, together with tests in Lib/ctypes/test ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610575&group_id=5470 From noreply at sourceforge.net Thu Dec 7 22:11:34 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 07 Dec 2006 13:11:34 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 14:29 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) Assigned to: Thomas Heller (theller) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2006-12-07 22:11 Message: Logged In: YES user_id=11105 Originator: NO Will do (although I would appreciate review from others too; I'm not exactly a BSD expert). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-07 20:15 Message: Logged In: YES user_id=21627 Originator: NO Thomas, can you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Fri Dec 8 00:59:19 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 07 Dec 2006 15:59:19 -0800 Subject: [Patches] [ python-Patches-1577756 ] whitespace in `svnversion .` Message-ID: Patches item #1577756, was opened at 2006-10-15 21:33 Message generated for change (Comment added) made by schmaller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1577756&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 >Status: Open >Resolution: Works For Me >Priority: 2 Private: No Submitted By: Daniel Str?nger (schmaller) Assigned to: Nobody/Anonymous (nobody) >Summary: whitespace in `svnversion .` Initial Comment: gcc (version 3.4.4) fails to compile Modules/getbuildinfo.c, since it works as documented and cuts the definition of SVNVERSION at the first newline and so there is an unbalanced double quote. citation of the gcc manpage: -D name=definition Predefine name as a macro, with definition definition. The contents of definition are tokenized and processed as if they appeared during translation phase three in a #define directive. In particular, the definition will be truncated by embedded newline characters. ---------------------------------------------------------------------- >Comment By: Daniel Str?nger (schmaller) Date: 2006-12-08 00:59 Message: Logged In: YES user_id=922469 Originator: YES Now I've found the error (and the solution). The culprit is the german version of svnversion: it returns the string "exportiert ". Notice the trailing space character. This leads to the following expansion for #> gcc -DSVNVERSION=\"`svnversion -n .`\" ... gcc -DSVNVERSION=\"exportiert \" ... and so SVNVERSION gets the value: <"exportiert>. To solve this problem, the value of the macro SVNVERSION must again be enclosed by double-quotes, as you can see in the patch file. Best regards, Daniel File Added: svnversion_whitespace.patch ---------------------------------------------------------------------- Comment By: Daniel Str?nger (schmaller) Date: 2006-10-15 23:12 Message: Logged In: YES user_id=922469 The patch don't fix it and i think it's not a problem of Python. I apologize for spamming the patch system. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1577756&group_id=5470 From noreply at sourceforge.net Fri Dec 8 08:13:04 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 07 Dec 2006 23:13:04 -0800 Subject: [Patches] [ python-Patches-1610575 ] C99 _Bool support for struct Message-ID: Patches item #1610575, was opened at 2006-12-07 05:37 Message generated for change (Comment added) made by chmod007 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610575&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Remahl (chmod007) Assigned to: Nobody/Anonymous (nobody) Summary: C99 _Bool support for struct Initial Comment: C99 adds the fundamental _Bool integer type (fundamental in the sense that it is not equivalent to or a composite of any other C type). Its size can vary from platform to platform; the only restriction imposed by the C standard is that it must be able to hold the values 0 or 1. Typically, sizeof _Bool is 1 or 4. A struct module user trying to parse a native C structure that contains a _Bool member faces a problem: struct does not have a format character for _Bool. One is forced to hardcode a size for bool (use a char or an int instead). This patch adds support for a new format character, 't', representing the fundamental type _Bool. It is handled sementically as representing pure booleans -- when packing a structure the truth value of the argument to be packed is used and when unpacking either True or False is always returned. For platforms that don't support _Bool, as well as in non-native mode, 't' packs as a single byte. Test cases are included, as well as a small change to the struct documentation. The patch modifies configure.in to check for _Bool support, and the patch includes the autogenerated configure and pyconfig.h.in files as well. I have tested the module on Mac OS X x86 (uses 1 byte for _Bool) and Mac OS X ppc (uses 4 bytes for _Bool). Ran regression suite. ---------------------------------------------------------------------- >Comment By: David Remahl (chmod007) Date: 2006-12-08 07:13 Message: Logged In: YES user_id=2135 Originator: YES Oops! I didn't intend for there to be any ctypes content in this patch (as indicated by the subject), but apparently I forgot to remove part of the ctypes section. I have uploaded a new patch without that part. Once this has been integrated, I'll upload a complete ctypes patch for consideration. File Added: bool struct patch-2.diff ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-07 20:09 Message: Logged In: YES user_id=11105 Originator: NO The patch is not complete or not correct. Either: - the part of that patch that changes Modules/_ctypes/_ctypes.c should be omitted because it does not contain a ctypes _Bool type - or complete support for a ctypes _Bool type (how would that be called? ctypes.c99_bool?) should be added, together with tests in Lib/ctypes/test ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610575&group_id=5470 From noreply at sourceforge.net Fri Dec 8 08:50:34 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 07 Dec 2006 23:50:34 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 13:29 Message generated for change (Comment added) made by chmod007 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) Assigned to: Thomas Heller (theller) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- Comment By: David Remahl (chmod007) Date: 2006-12-08 07:50 Message: Logged In: YES user_id=2135 Originator: NO # Does this work (without the gcc fallback) on other *BSD systems too? I don't know, but it doesn't work on Darwin (which already has a custom method through macholib). ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-07 21:11 Message: Logged In: YES user_id=11105 Originator: NO Will do (although I would appreciate review from others too; I'm not exactly a BSD expert). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-07 19:15 Message: Logged In: YES user_id=21627 Originator: NO Thomas, can you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Fri Dec 8 21:32:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 08 Dec 2006 12:32:22 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 14:29 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) Assigned to: Thomas Heller (theller) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2006-12-08 21:32 Message: Logged In: YES user_id=11105 Originator: NO I have tested the patch on FreeBSD 6.0 and (after extending the check to test for sys.platform.startswith("openbsd")) on OpenBSD 3.9 and it works fine. find_library("c") now returns libc.so.6 on FreeBSD 6.0, and libc.so.39.0 in OpenBSD 3.9, while it returned 'None' before on both machines. ---------------------------------------------------------------------- Comment By: David Remahl (chmod007) Date: 2006-12-08 08:50 Message: Logged In: YES user_id=2135 Originator: NO # Does this work (without the gcc fallback) on other *BSD systems too? I don't know, but it doesn't work on Darwin (which already has a custom method through macholib). ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-07 22:11 Message: Logged In: YES user_id=11105 Originator: NO Will do (although I would appreciate review from others too; I'm not exactly a BSD expert). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-07 20:15 Message: Logged In: YES user_id=21627 Originator: NO Thomas, can you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Fri Dec 8 21:47:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 08 Dec 2006 12:47:25 -0800 Subject: [Patches] [ python-Patches-1577756 ] whitespace in `svnversion .` Message-ID: Patches item #1577756, was opened at 2006-10-15 19:33 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1577756&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 >Status: Closed >Resolution: Fixed Priority: 2 Private: No Submitted By: Daniel Str?nger (schmaller) >Assigned to: Georg Brandl (gbrandl) Summary: whitespace in `svnversion .` Initial Comment: gcc (version 3.4.4) fails to compile Modules/getbuildinfo.c, since it works as documented and cuts the definition of SVNVERSION at the first newline and so there is an unbalanced double quote. citation of the gcc manpage: -D name=definition Predefine name as a macro, with definition definition. The contents of definition are tokenized and processed as if they appeared during translation phase three in a #define directive. In particular, the definition will be truncated by embedded newline characters. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-08 20:47 Message: Logged In: YES user_id=849994 Originator: NO German output shouldn't have happened. It turns out that LANG=C isn't the correct env var to force English output, now using LC_ALL=C in rev. 52970, 52971 (2.5). ---------------------------------------------------------------------- Comment By: Daniel Str?nger (schmaller) Date: 2006-12-07 23:59 Message: Logged In: YES user_id=922469 Originator: YES Now I've found the error (and the solution). The culprit is the german version of svnversion: it returns the string "exportiert ". Notice the trailing space character. This leads to the following expansion for #> gcc -DSVNVERSION=\"`svnversion -n .`\" ... gcc -DSVNVERSION=\"exportiert \" ... and so SVNVERSION gets the value: <"exportiert>. To solve this problem, the value of the macro SVNVERSION must again be enclosed by double-quotes, as you can see in the patch file. Best regards, Daniel File Added: svnversion_whitespace.patch ---------------------------------------------------------------------- Comment By: Daniel Str?nger (schmaller) Date: 2006-10-15 21:12 Message: Logged In: YES user_id=922469 The patch don't fix it and i think it's not a problem of Python. I apologize for spamming the patch system. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1577756&group_id=5470 From noreply at sourceforge.net Fri Dec 8 21:55:24 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 08 Dec 2006 12:55:24 -0800 Subject: [Patches] [ python-Patches-1609282 ] #1603424 subprocess.py wrongly claims 2.2 compatibility. Message-ID: Patches item #1609282, was opened at 2006-12-05 15:16 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609282&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Robert Carr (racarr) Assigned to: Nobody/Anonymous (nobody) Summary: #1603424 subprocess.py wrongly claims 2.2 compatibility. Initial Comment: Simple fix restoring 2.2 compatibility in subprocess.py. This makes more sense than a list comprehension or constructing sets in my opinion even ignoring the bug. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-08 20:55 Message: Logged In: YES user_id=849994 Originator: NO This patch changes semantics: if two names refer to the same fd, it is attempted to be closed twice. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609282&group_id=5470 From noreply at sourceforge.net Sat Dec 9 09:56:06 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 09 Dec 2006 00:56:06 -0800 Subject: [Patches] [ python-Patches-1609407 ] traceback on exit if syslog handler fails to initialize Message-ID: Patches item #1609407, was opened at 2006-12-05 16:43 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609407&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jeremy Katz (katzj) >Assigned to: Vinay Sajip (vsajip) Summary: traceback on exit if syslog handler fails to initialize Initial Comment: Syslog loggers can fail to initialize if the syslogd has crashed; this then leads to a traceback on exit as self.unixsocket isn't set. Attached patch changes around the order to avoid this. Related to http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=206474 ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-09 08:56 Message: Logged In: YES user_id=849994 Originator: NO Assigning to Vinay. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609407&group_id=5470 From noreply at sourceforge.net Sat Dec 9 10:10:51 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 09 Dec 2006 01:10:51 -0800 Subject: [Patches] [ python-Patches-1608579 ] Race condition in os.makedirs Message-ID: Patches item #1608579, was opened at 2006-12-04 16:52 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608579&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: James Bowes (jbowes) Assigned to: Nobody/Anonymous (nobody) Summary: Race condition in os.makedirs Initial Comment: os.makedirs first checks if the non-leaf part of the path exists, and then calls makedirs on this part of the path to create it. If the path is created in between the call to path.exists, and the recursive call to os.makedirs, then os.makedirs will raise an exception. The attached patch catches this exception, and then tests if the exception is caused by a the file/directory's existence. If it is, the exception is ignored. If not, it is rasied. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-09 09:10 Message: Logged In: YES user_id=849994 Originator: NO Committed as rev. 52972, 52973 (2.5). ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:58 Message: Logged In: YES user_id=161998 Originator: NO looks good to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1608579&group_id=5470 From noreply at sourceforge.net Sat Dec 9 10:24:11 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 09 Dec 2006 01:24:11 -0800 Subject: [Patches] [ python-Patches-1239890 ] Prevent race condition in os.makedirs Message-ID: Patches item #1239890, was opened at 2005-07-17 19:42 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1239890&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Out of Date Priority: 5 Private: No Submitted By: Nir Soffer (nirs) Assigned to: Nobody/Anonymous (nobody) Summary: Prevent race condition in os.makedirs Initial Comment: makedirs check if a directory exists, and call mkdir to create it. In rare cases, the directory might have been created by another process or thread in the time passed from the exists() call to the mkdir() call. To prevent this rare race condition, use try: expect: to create the directory, then ignore "File exists" errors, which mean the directory was there. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-09 09:24 Message: Logged In: YES user_id=849994 Originator: NO Something similar has been committed as patch #1608267. ---------------------------------------------------------------------- Comment By: Nir Soffer (nirs) Date: 2005-12-05 12:46 Message: Logged In: YES user_id=832344 Sorry, here is the patch. ---------------------------------------------------------------------- Comment By: Nir Soffer (nirs) Date: 2005-12-05 01:21 Message: Logged In: YES user_id=832344 Please review this new patch, tested with current 2.5 code. I factored duplicate code in makedirs and removedirs into private _splits function, kind of the "super" version of os.path.split for the "super" directory utilities. I also simplified the test code for makedirs, and added more test cases. ---------------------------------------------------------------------- Comment By: Nir Soffer (nirs) Date: 2005-12-05 01:08 Message: Logged In: YES user_id=832344 I deleted both patches as they are both wrong: The patch against 2.4.1 will not raise OSError when trying to create existing directories. The simpler code for 2.5 will not work because os.path.normpath is not safe to use if the path contain symbolic links. ---------------------------------------------------------------------- Comment By: Nir Soffer (nirs) Date: 2005-12-02 18:11 Message: Logged In: YES user_id=832344 The 2.5 code is only a proof of concept, don't use it as is :-) All the directories are handled in the same way - ignore OSError for existing directories, which is not compatible with the docs, that say it should raise an OSError for the last directory. This has to be fixed of course. I'll get 2.5 development code and submit a real working patch. ---------------------------------------------------------------------- Comment By: Richard Barran (rbarran) Date: 2005-12-02 16:07 Message: Logged In: YES user_id=1207189 Hi, Your patch "makedirs.py" for python 2.5 will fail under Windows if a drive letter is specified: >>> os.getcwd() 'C:\\Temp\\makedirs' >>> os.listdir('.') ['makedirs.py', 'makedirs.pyc', 'test.py'] >>> makedirs.makedirs('c:/temp/makedirs/a/b') >>> os.listdir('.') ['makedirs.py', 'makedirs.pyc', 'temp', 'test.py'] It will create the following path: C:\Temp\makedirs\temp\makedirs\a\b Also I ran it through the test suite (lib\test\test_os.py) and it failed one test: ====================================================================== FAIL: test_makedir (__main__.MakedirTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "test_os.py", line 327, in test_makedir self.failUnlessRaises(OSError, os.makedirs, os.curdir) AssertionError: OSError not raised ---------------------------------------------------------------------- I haven't looked into *why* this happens - I'll dig a bit deeper into this subject sometime next week. HTH, Richard ---------------------------------------------------------------------- Comment By: Nir Soffer (nirs) Date: 2005-07-17 21:13 Message: Logged In: YES user_id=832344 Here is another patch, using simpler design, maybe for 2.5? The attached patch will try once to create each leaf in the path, ignoring existing leafs. This should work for most cases. If one run application that can delete its own directories, he should use proper locking. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-07-17 20:13 Message: Logged In: YES user_id=1188172 http://sourceforge.net/tracker/index.php?func=detail&aid=1223238&group_id=5470&atid=105470 ---------------------------------------------------------------------- Comment By: Nir Soffer (nirs) Date: 2005-07-17 20:06 Message: Logged In: YES user_id=832344 I can't find such bug or patch. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-07-17 19:57 Message: Logged In: YES user_id=1188172 See bug #1223238. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1239890&group_id=5470 From noreply at sourceforge.net Sat Dec 9 10:26:59 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 09 Dec 2006 01:26:59 -0800 Subject: [Patches] [ python-Patches-1314067 ] os.makedirs - robust against partial path Message-ID: Patches item #1314067, was opened at 2005-10-05 17:08 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1314067&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Jim Jewett (jimjjewett) Assigned to: Nobody/Anonymous (nobody) Summary: os.makedirs - robust against partial path Initial Comment: os.py function makedirs is intended to create a directory, including any required parent directories. Unfortunately, at least on windows, it fails is some of those parent directories already exist. This patch says "if the directory I'm about to create is already an existing directory, then pretend I succeeded and continue with the next step." ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-09 09:26 Message: Logged In: YES user_id=849994 Originator: NO I agree with rbarran. Closing as Rejected. ---------------------------------------------------------------------- Comment By: Richard Barran (rbarran) Date: 2005-12-05 12:39 Message: Logged In: YES user_id=1207189 I think that the current behavior of the function is correct. It tries to mimic os.mkdir - if the dir to create ("c" in your example) already exists, it will raise an error. However, errors on intermediate dirs are ignored. This makes sense, but I think the documentation is not clear on this (or maybe I'm just slow :-) BTW - there's another patch ([ 1239890 ] Prevent race condition in os.makedirs) to rewrite the whole function and it might make your patch in its current form obsolete. Maybe your work can be merged into this other patch? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2005-12-02 20:11 Message: Logged In: YES user_id=764593 slight misdiagnosis on my part -- it only fails if the *entire* directory tree already exists. """ >>> os.makedirs('c:/temp/a/b/c') >>> os.makedirs('c:/temp/a/b/c') Traceback (most recent call last): File "", line 1, in -toplevel- os.makedirs('c:/temp/a/b/c') File "C:\Python24\lib\orig_os.py", line 159, in makedirs mkdir(name, mode) OSError: [Errno 17] File exists: 'c:/temp/a/b/c' """ My use case was generating java files in the proper package - and possibly regenerating them if something changed. I want to put them in the right directory, which will usually (but not always) already exist. The patch still works, but I now wonder if it might be better to put a guard at the top along the lines of "if path.exists(name): return" (or something fancier to ensure that it is a directory with appropriate permissions). ---------------------------------------------------------------------- Comment By: Richard Barran (rbarran) Date: 2005-12-01 16:39 Message: Logged In: YES user_id=1207189 Hi, Could you provide some example code that shows up the error? If I understand you correctly, the following should fail: import os os.mkdir('c:/temp/a') os.makedirs('c:/temp/a/b/c') But it works fine on my WinXP pro SP2 machine. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1314067&group_id=5470 From noreply at sourceforge.net Sat Dec 9 17:51:04 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 09 Dec 2006 08:51:04 -0800 Subject: [Patches] [ python-Patches-1540869 ] CodeContext visibility Message-ID: Patches item #1540869, was opened at 2006-08-15 22:20 Message generated for change (Comment added) made by taleinat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1540869&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: Python 2.6 Status: Open Resolution: Later Priority: 5 Private: No Submitted By: Jim Jewett (jimjjewett) Assigned to: Kurt B. Kaiser (kbk) Summary: CodeContext visibility Initial Comment: CodeContext hardcodes several constants; these two in particular make it difficult to visually separate the "context" from the top of the regular code. Ideally, the measurement would change to something like em or ex, but ... this was an improvement. Note that the separation is explicitly there in the code already; it just doesn't work with the current constants. If the Relesae Manager decides to delay it to 2.6 (or backport to 2.4), I won't object. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-12-09 18:51 Message: Logged In: YES user_id=1330769 Originator: NO Increasing the vertical separation is a possibility, but I think this is mostly a matter of taste. Personally, I prefer to differentiate between the code and context with a different backgroud than to have a large blank separator. (Note that the background and foreground colors of the context are configurable.) In any case, just increasing the border ruins the horizontal alignment. This should be done with 'pady' or by adding a padding frame below the context window. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-08-16 20:19 Message: Logged In: YES user_id=764593 ahh ... I hadn't even noticed the (effectively one- character) horizontal change. The change I aimed for (which I thought was the goal of the pad frame) was a slight *vertical* separation between the parts. So definately not until 2.6, and hopefully the eventual replacement will be better than what I posted here. Thank you for the clarification. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-08-16 20:18 Message: Logged In: YES user_id=764593 ahh ... I hadn't even noticed the (effectively one- character) horizontal change. The change I aimed for (which I thought was the goal of the pad frame) was a slight *vertical* separation between the parts. So definately not until 2.6, and hopefully the eventual replacement will be better than what I posted here. Thank you for the clarification. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-08-16 18:40 Message: Logged In: YES user_id=1330769 I originally wrote the code with those constants. I used those constants because I specifically wanted to line up the CodeContext code with the code in the main window, for easier reading. Your patch does the exact opposite - it pushes the CodeContext code way to the right. IMO, since the purpose of CodeContext is to have the current scope easily seen, having the code aligned is vital. You may prefer dis-alignment because it's easier to tell CodeContext apart, but that's just a personal preference, and IMO not something that should be the default behavior of IDLE. Personally, I find that CodeContext's different background color is more than enough to distinguish it from the main editor window. CodeContext's background and foreground colors are configurable (in config-extensions), you can always choose colors you like better. Tal ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1540869&group_id=5470 From noreply at sourceforge.net Sun Dec 10 00:48:49 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 09 Dec 2006 15:48:49 -0800 Subject: [Patches] [ python-Patches-1597850 ] Cross compiling patches for MINGW Message-ID: Patches item #1597850, was opened at 2006-11-16 16:57 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Han-Wen Nienhuys (hanwen) Assigned to: Nobody/Anonymous (nobody) Summary: Cross compiling patches for MINGW Initial Comment: Hello, attached tarbal is a patch bomb of 32 patches against python 2.5, that we lilypond developers use for crosscompiling python. The patches were originally written by Jan Nieuwenhuizen, my codeveloper. These patches have been tested with Linux/x86, linux/x64 and macos 10.3 as build host and linux-{ppc,x86,x86_64}, freebsd, mingw as target platform. All packages at lilypond.org/install/ except for darwin contain the x-compiled python. Each patch is prefixed with a small comment, but for reference, I include a snippet from the readme. It would be nice if at least some of the patches were included. In particular, I think that X-compiling is a common request, so it warrants inclusion. Basically, what we do is override autoconf and Makefile settings through setting enviroment variables. **README section** Cross Compiling --------------- Python can be cross compiled by supplying different --build and --host parameters to configure. Python is compiled on the "build" system and executed on the "host" system. Cross compiling python requires a native Python on the build host, and a natively compiled tool `Pgen'. Before cross compiling, Python must first be compiled and installed on the build host. The configure script will use `cc' and `python', or environment variables CC_FOR_BUILD or PYTHON_FOR_BUILD, eg: CC_FOR_BUILD=gcc-3.3 \ PYTHON_FOR_BUILD=python2.4 \ .../configure --build=i686-linux --host=i586-mingw32 Cross compiling has been tested under linux, mileage may vary for other platforms. A few reminders on using configure to cross compile: - Cross compile tools must be in PATH, - Cross compile tools must be prefixed with the host type (ie i586-mingw32-gcc, i586-mingw32-ranlib, ...), - CC, CXX, AR, and RANLIB must be undefined when running configure, they will be auto-detected. If you need a cross compiler, Debian ships several several (eg: avr, m68hc1x, mingw32), while dpkg-cross easily creates others. Otherwise, check out Dan Kegel's crosstool: http://www.kegel.com/crosstool . ---------------------------------------------------------------------- >Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-09 23:48 Message: Logged In: YES user_id=161998 Originator: YES With cross.patch I've been able to build a working freebsd python on linux. Since you had little problems with the X-compile patches, I'm resubmitting those first. I'd like to give our (admittedly: oddball) mingw version another go when the X-compile patches are in python SVN. Regarding your comments: * what would be a better to import the SO setting? the most reliable way to get something out of a makefile into python is VAR=foo export VAR .. os.environ['VAR'] this doesn't introduce any fragility in parsing/expanding/(un)quoting, so it's actually pretty good. Right now, I'm overriding sysconfig wholesale in setup.py with a sysconfig._config_vars.update (os.environ) but I'm not sure that this affects the settings in build_ext.py. A freebsd -> linux compile does not touch that code, so if you dislike it, we can leave it out. * I've documented the .x extension File Added: cross.patch ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-06 20:12 Message: Logged In: YES user_id=21627 Originator: NO One more note: it would be best if the patches were against the subversion trunk. They won't be included in the 2.5 maintenance branch (as they are a new feature), so they need to be ported to the trunk, anyway. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-06 20:06 Message: Logged In: YES user_id=21627 Originator: NO I'll add my comments as I go through the patches. cab1e7d1e54d14a8aab52f0c3b3073c93f75d4fc: - why is there now a mingw32msvc2 platform? If the target is mingw (rather than Cygwin), I'd expect that the target is just Win32/Windows, and that all symbolic constants provided be usable across all Win32 Pythons. - why is h2py run for /usr/include/netinet/in.h? Shouldn't it operate on a target header file? - please include any plat-* files that you generate in the patch. - why do you need dl_nt.c in Modules? Please make it use the one from PC (consider updating the comment about calling initall) b52dbbbbc3adece61496b161d8c22599caae2311 - please combine all patches adding support for __MINGW32__ into a single one. Why is anything needed here at all? I thought Python compiles already with mingw32 (on Windows)? - what is the exclusion of freezing for? 059af829d362b10bb5921367c93a56dbb51ef31b - Why are you taking timeval from winsock2.h? It should come from sys/time.h, and does in my copy of Debian mingw32-runtime. 6a742fb15b28564f9a1bc916c76a28dc672a9b2c - Why are these changes needed? It's Windows, and that is already supported. a838b4780998ef98ae4880c3916274d45b661c82 - Why doesn't that already work on Windows+cygwin+mingw32? f452fe4b95085d8c1ba838bf302a6a48df3c1d31 - I think this should target msvcr71.dll, not msvcrt.dll Please also combine the cross-compilation patches into a single one. - there is no need to provide pyconfig.h.in changes; I'll regenerate that, anyway. 9c022e407c366c9f175e9168542ccc76eae9e3f0 - please integrate those into the large AC_CHECK_FUNCS that already exists 540684d696df6057ee2c9c4e13e33fe450605ffa - Why are you stripping -Wl? 64f5018e975419b2d37c39f457c8732def3288df - Try getting SO from the Makefile, not from the environment (I assume this is also meant to support true distutils packages some day). 7a4e50fb1cf5ff3481aaf7515a784621cbbdac6c - again: what is the "mingw" platform? 7d3a45788a0d83608d10e5c0a34f08b426d62e92 - is this really necessary? I suggest to drop it 23a2dd14933a2aee69f7cdc9f838e4b9c26c1eea - don't include bits/time.h; it's not meant for direct inclusion 6689ca9dea07afbe8a77b7787a5c4e1642f803a1 - what's a .x file? ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-25 15:12 Message: Logged In: YES user_id=161998 Originator: YES I've sent the agreement by snailmail. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2006-11-17 19:57 Message: Logged In: YES user_id=1368960 Originator: NO I do not mind either. I've just signed and faxed contrib-form.html. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 00:33 Message: Logged In: YES user_id=161998 Originator: YES note that not all of the patch needs to go in its current form. In particular, setup.py should be much more clever to look into build-root for finding libs and include files. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 00:32 Message: Logged In: YES user_id=161998 Originator: YES I don't mind, and I expect Jan won't have a problem either. What's the procedure: do we send the disclaimer first, or do you do the review, or does everything happen in parallel? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-16 21:47 Message: Logged In: YES user_id=21627 Originator: NO Would you and Jan Nieuwenhuizen be willing to sign the contributor agreement, at http://www.python.org/psf/contrib.html I haven't reviewed the patch yet; if they can be integrated, that will only happen in the trunk (i.e. not for 2.5.x). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 From noreply at sourceforge.net Sun Dec 10 00:50:15 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 09 Dec 2006 15:50:15 -0800 Subject: [Patches] [ python-Patches-1597850 ] Cross compiling patches for MINGW Message-ID: Patches item #1597850, was opened at 2006-11-16 16:57 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Han-Wen Nienhuys (hanwen) Assigned to: Nobody/Anonymous (nobody) Summary: Cross compiling patches for MINGW Initial Comment: Hello, attached tarbal is a patch bomb of 32 patches against python 2.5, that we lilypond developers use for crosscompiling python. The patches were originally written by Jan Nieuwenhuizen, my codeveloper. These patches have been tested with Linux/x86, linux/x64 and macos 10.3 as build host and linux-{ppc,x86,x86_64}, freebsd, mingw as target platform. All packages at lilypond.org/install/ except for darwin contain the x-compiled python. Each patch is prefixed with a small comment, but for reference, I include a snippet from the readme. It would be nice if at least some of the patches were included. In particular, I think that X-compiling is a common request, so it warrants inclusion. Basically, what we do is override autoconf and Makefile settings through setting enviroment variables. **README section** Cross Compiling --------------- Python can be cross compiled by supplying different --build and --host parameters to configure. Python is compiled on the "build" system and executed on the "host" system. Cross compiling python requires a native Python on the build host, and a natively compiled tool `Pgen'. Before cross compiling, Python must first be compiled and installed on the build host. The configure script will use `cc' and `python', or environment variables CC_FOR_BUILD or PYTHON_FOR_BUILD, eg: CC_FOR_BUILD=gcc-3.3 \ PYTHON_FOR_BUILD=python2.4 \ .../configure --build=i686-linux --host=i586-mingw32 Cross compiling has been tested under linux, mileage may vary for other platforms. A few reminders on using configure to cross compile: - Cross compile tools must be in PATH, - Cross compile tools must be prefixed with the host type (ie i586-mingw32-gcc, i586-mingw32-ranlib, ...), - CC, CXX, AR, and RANLIB must be undefined when running configure, they will be auto-detected. If you need a cross compiler, Debian ships several several (eg: avr, m68hc1x, mingw32), while dpkg-cross easily creates others. Otherwise, check out Dan Kegel's crosstool: http://www.kegel.com/crosstool . ---------------------------------------------------------------------- >Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-09 23:50 Message: Logged In: YES user_id=161998 Originator: YES this is a patch against a SVN checkout of last week. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-09 23:48 Message: Logged In: YES user_id=161998 Originator: YES With cross.patch I've been able to build a working freebsd python on linux. Since you had little problems with the X-compile patches, I'm resubmitting those first. I'd like to give our (admittedly: oddball) mingw version another go when the X-compile patches are in python SVN. Regarding your comments: * what would be a better to import the SO setting? the most reliable way to get something out of a makefile into python is VAR=foo export VAR .. os.environ['VAR'] this doesn't introduce any fragility in parsing/expanding/(un)quoting, so it's actually pretty good. Right now, I'm overriding sysconfig wholesale in setup.py with a sysconfig._config_vars.update (os.environ) but I'm not sure that this affects the settings in build_ext.py. A freebsd -> linux compile does not touch that code, so if you dislike it, we can leave it out. * I've documented the .x extension File Added: cross.patch ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-06 20:12 Message: Logged In: YES user_id=21627 Originator: NO One more note: it would be best if the patches were against the subversion trunk. They won't be included in the 2.5 maintenance branch (as they are a new feature), so they need to be ported to the trunk, anyway. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-06 20:06 Message: Logged In: YES user_id=21627 Originator: NO I'll add my comments as I go through the patches. cab1e7d1e54d14a8aab52f0c3b3073c93f75d4fc: - why is there now a mingw32msvc2 platform? If the target is mingw (rather than Cygwin), I'd expect that the target is just Win32/Windows, and that all symbolic constants provided be usable across all Win32 Pythons. - why is h2py run for /usr/include/netinet/in.h? Shouldn't it operate on a target header file? - please include any plat-* files that you generate in the patch. - why do you need dl_nt.c in Modules? Please make it use the one from PC (consider updating the comment about calling initall) b52dbbbbc3adece61496b161d8c22599caae2311 - please combine all patches adding support for __MINGW32__ into a single one. Why is anything needed here at all? I thought Python compiles already with mingw32 (on Windows)? - what is the exclusion of freezing for? 059af829d362b10bb5921367c93a56dbb51ef31b - Why are you taking timeval from winsock2.h? It should come from sys/time.h, and does in my copy of Debian mingw32-runtime. 6a742fb15b28564f9a1bc916c76a28dc672a9b2c - Why are these changes needed? It's Windows, and that is already supported. a838b4780998ef98ae4880c3916274d45b661c82 - Why doesn't that already work on Windows+cygwin+mingw32? f452fe4b95085d8c1ba838bf302a6a48df3c1d31 - I think this should target msvcr71.dll, not msvcrt.dll Please also combine the cross-compilation patches into a single one. - there is no need to provide pyconfig.h.in changes; I'll regenerate that, anyway. 9c022e407c366c9f175e9168542ccc76eae9e3f0 - please integrate those into the large AC_CHECK_FUNCS that already exists 540684d696df6057ee2c9c4e13e33fe450605ffa - Why are you stripping -Wl? 64f5018e975419b2d37c39f457c8732def3288df - Try getting SO from the Makefile, not from the environment (I assume this is also meant to support true distutils packages some day). 7a4e50fb1cf5ff3481aaf7515a784621cbbdac6c - again: what is the "mingw" platform? 7d3a45788a0d83608d10e5c0a34f08b426d62e92 - is this really necessary? I suggest to drop it 23a2dd14933a2aee69f7cdc9f838e4b9c26c1eea - don't include bits/time.h; it's not meant for direct inclusion 6689ca9dea07afbe8a77b7787a5c4e1642f803a1 - what's a .x file? ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-25 15:12 Message: Logged In: YES user_id=161998 Originator: YES I've sent the agreement by snailmail. ---------------------------------------------------------------------- Comment By: Jan Nieuwenhuizen (janneke-sf) Date: 2006-11-17 19:57 Message: Logged In: YES user_id=1368960 Originator: NO I do not mind either. I've just signed and faxed contrib-form.html. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 00:33 Message: Logged In: YES user_id=161998 Originator: YES note that not all of the patch needs to go in its current form. In particular, setup.py should be much more clever to look into build-root for finding libs and include files. ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-11-17 00:32 Message: Logged In: YES user_id=161998 Originator: YES I don't mind, and I expect Jan won't have a problem either. What's the procedure: do we send the disclaimer first, or do you do the review, or does everything happen in parallel? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-16 21:47 Message: Logged In: YES user_id=21627 Originator: NO Would you and Jan Nieuwenhuizen be willing to sign the contributor agreement, at http://www.python.org/psf/contrib.html I haven't reviewed the patch yet; if they can be integrated, that will only happen in the trunk (i.e. not for 2.5.x). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1597850&group_id=5470 From noreply at sourceforge.net Sun Dec 10 04:20:05 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 9 Dec 2006 19:20:05 -0800 Subject: [Patches] [ python-Patches-858318 ] modsupport does not use va_copy when available Message-ID: <200612100320.kBA3K5Ce023733@sc8-sf-db2-new-b.sourceforge.net> Patches item #858318, was opened at 2003-12-11 07:30 Message generated for change (Comment added) made by sf-robot You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=858318&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 >Status: Closed Resolution: Out of Date Priority: 5 Private: No Submitted By: benson margulies (benson_basis) Assigned to: Nobody/Anonymous (nobody) Summary: modsupport does not use va_copy when available Initial Comment: It appears that the gcc 3.2.2 on SusE linux AMD 64-bit requires the use of va_copy. The attached patch uses va_copy whenever it is #defined. ---------------------------------------------------------------------- >Comment By: SourceForge Robot (sf-robot) Date: 2006-12-09 19:20 Message: Logged In: YES user_id=1312539 Originator: NO This Tracker item was closed automatically by the system. It was previously set to a Pending status, and the original submitter did not respond within 14 days (the time period specified by the administrator of this Tracker). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-25 07:45 Message: Logged In: YES user_id=21627 Originator: NO Is this patch still needed? Nobody has complained about a problem with va-lists on AMD64 since. Tentatively marking it out-of-date. ---------------------------------------------------------------------- Comment By: benson margulies (benson_basis) Date: 2003-12-11 11:40 Message: Logged In: YES user_id=876734 I've marked this 2.4, but someone (SuSe?) might like it sooner. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=858318&group_id=5470 From noreply at sourceforge.net Sun Dec 10 21:14:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 10 Dec 2006 12:14:42 -0800 Subject: [Patches] [ python-Patches-1612746 ] Enhanced tabbed pane widget Message-ID: Patches item #1612746, was opened at 2006-12-10 22:14 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1612746&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Nobody/Anonymous (nobody) Summary: Enhanced tabbed pane widget Initial Comment: tabbedPage.py is a replacement for tabpage.py. Other than the test code, the code has been almost completely re-written. Enhancements over tabpage.py: * Tab panes with several rows of tabs (optional) * Dynamic tab rows (optional) - tabs are divided into rows as needed, currently according to a configurable maximum number of tabs per row * Simpler, more Pythonic design and API * Two placement mechanisms for the inner pages - grid.lift and grid/remove. Each creates different behavior, and I couldn't decide between them, so I made it configurable :) * Continues to support dynamic addition/removal of pages * Additional comments and documentation, code cleanup Also included in the patch are the required changes in configDialog.py for it to use tabbedPages.py, along with some minor widget padding fixes. (tabpage.py is no longer needed, though the patch doesn't say so for some reason) BTW, this is required for the IDLE extension configuration dialog I've written, for which I will post a patch soon. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1612746&group_id=5470 From noreply at sourceforge.net Sun Dec 10 21:16:24 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 10 Dec 2006 12:16:24 -0800 Subject: [Patches] [ python-Patches-1612746 ] Enhanced tabbed pane widget Message-ID: Patches item #1612746, was opened at 2006-12-10 22:14 Message generated for change (Comment added) made by taleinat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1612746&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Open Resolution: None >Priority: 3 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Nobody/Anonymous (nobody) Summary: Enhanced tabbed pane widget Initial Comment: tabbedPage.py is a replacement for tabpage.py. Other than the test code, the code has been almost completely re-written. Enhancements over tabpage.py: * Tab panes with several rows of tabs (optional) * Dynamic tab rows (optional) - tabs are divided into rows as needed, currently according to a configurable maximum number of tabs per row * Simpler, more Pythonic design and API * Two placement mechanisms for the inner pages - grid.lift and grid/remove. Each creates different behavior, and I couldn't decide between them, so I made it configurable :) * Continues to support dynamic addition/removal of pages * Additional comments and documentation, code cleanup Also included in the patch are the required changes in configDialog.py for it to use tabbedPages.py, along with some minor widget padding fixes. (tabpage.py is no longer needed, though the patch doesn't say so for some reason) BTW, this is required for the IDLE extension configuration dialog I've written, for which I will post a patch soon. ---------------------------------------------------------------------- >Comment By: Tal Einat (taleinat) Date: 2006-12-10 22:16 Message: Logged In: YES user_id=1330769 Originator: YES Typo - the file is tabbedPages.py ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1612746&group_id=5470 From noreply at sourceforge.net Mon Dec 11 05:36:47 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 10 Dec 2006 20:36:47 -0800 Subject: [Patches] [ python-Patches-1087418 ] long int bitwise ops speedup (patch included) Message-ID: Patches item #1087418, was opened at 2004-12-18 06:22 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1087418&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: None Status: Open Resolution: None Priority: 4 Private: No Submitted By: Gregory Smith (gregsmith) Assigned to: Tim Peters (tim_one) Summary: long int bitwise ops speedup (patch included) Initial Comment: The 'inner loop' for applying bitwise ops to longs is quite inefficient. The improvement in the attached diff is - 'a' is never shorter than 'b' (result: only test 1 loop index condition instead of 3) - each operation ( & | ^ ) has its own loop, instead of switch inside loop - I found that, when this is done, a lot of things can be simplified, resulting in further speedup, and the resulting code is not very much longer than before (my libpython2.4.dll .text got 140 bytes longer). Operations on longs of a few thousand bits appear to be 2 ... 2.5 times faster with this patch. I'm not 100% sure the code is right, but it passes test_long.py, anyway. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-11 05:36 Message: Logged In: YES user_id=21627 Originator: NO Tim, what is your view on this patch? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2006-05-25 21:54 Message: Logged In: YES user_id=31435 Assigned to me, changed Category to Performance. ---------------------------------------------------------------------- Comment By: Gregory Smith (gregsmith) Date: 2005-02-11 04:45 Message: Logged In: YES user_id=292741 I started by just factoring out the inner switch loop. But then it becomes evident that when op = '^', you always have maska == maskb, so there's no point in doing the ^mask at all. And when op == '|', then maska==maskb==0. So likewise. And if you put a check in so that len(a) >= len(b), then the calculation of len_z can be simplified. It also becomes easy to break the end off the loops, so that, say, or'ing a small number with a really long becomes mostly a copy. etc. It's was just a series of small simple changes following from the refactoring of the loop/switch. I see a repeatable 1.5 x speedup at 300 bits, which I think is significant (I wasn't using negative #s, which of course have their own extra overhead). The difference should be even higher on CPUs that don't have several 100 mW of branch-prediction circuitry. One use case is that you can simulate an array of hundreds or thousands of simple 1-bit processors in pure python using long operations, and get very good performance, even better with this fix. This app involves all logical ops, with the occasional shift. IMHO, I don't think the changed code is more complex; it's a little longer, but it's more explicit in what is really being done, and it doesn't roll together 3 cases, which don't really have that much in common, for the sake of brevity. It wasn't obvious to me about the masks being redundant until after I did the factoring, and this is my point - rolling it together hides that. The original author may not have noticed the redundancy. I see a lot of effort being expended on very complex multiply operations, why should the logical ops be left behind for the sake of a few lines? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-01-07 07:54 Message: Logged In: YES user_id=80475 Patch Review ------------ On Windows using MSC 6.0, I could only reproduce about a small speedup at around 300 bits. While the patch is short, it adds quite a bit of complexity to the routine. Its correctness is not self-evident or certain. Even if correct, it is likely to encumber future maintenance. Unless you have important use cases and feel strongly about it, I think this one should probably not go in. An alternative to submit a patch that limits its scope to factoring out the innermost switch/case. I tried that and found that the speedup is microscopic. I suspect that that one unpredictable branch is not much of a bottleneck. More time is likely spent on creating z. ---------------------------------------------------------------------- Comment By: Gregory Smith (gregsmith) Date: 2005-01-03 20:54 Message: Logged In: YES user_id=292741 I originally timed this on a cygwin system, I've since found that cygwin timings tend to be strange and possibly misleading. On a RH8 system, I'm seeing speedup of x3.5 with longs of ~1500 bits and larger, and x1.5 speedup with only about 300 bits. Times were measured with timeit.Timer( 'a|b', 'a=...; b=...') Increase in .text size is likewise about 120 bytes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1087418&group_id=5470 From noreply at sourceforge.net Mon Dec 11 05:38:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 10 Dec 2006 20:38:37 -0800 Subject: [Patches] [ python-Patches-1088832 ] acknowledge signals in non-main threads Message-ID: Patches item #1088832, was opened at 2004-12-21 06:15 Message generated for change (Settings changed) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1088832&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.3 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Andrew Langmead (langmead) Assigned to: Anthony Baxter (anthonybaxter) Summary: acknowledge signals in non-main threads Initial Comment: When a thread under Python 2.3 produces a synchronous signal (like a SIGSEGV) the signal is blocked and does not get handled by the Python interpreter. Python 2.4 corrected this behavior and signals sent to a thread are acknowledged, and then handled the signal default behavior unless overridden with a signal handler. (in which case the python signal handler is run on the main thread.) This is a patch for Python 2.3 for proper thread/signal behavior similar to . (This isn't one of the patches attached to the patch ticket, its the changes that mwh made in the CVS revisions listed, plus the further corrections in later revisions of the affected files.) ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-11 05:38 Message: Logged In: YES user_id=21627 Originator: NO Clearly, the 2.3 branch is closed now, so I'm rejecting this patch. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2004-12-21 15:54 Message: Logged In: YES user_id=6656 I'd be a little bit scared about applying this patch to the 2.3 branch, to be honest. I wonder what Anthony thinks? ---------------------------------------------------------------------- Comment By: Andrew Langmead (langmead) Date: 2004-12-21 06:17 Message: Logged In: YES user_id=119306 Since mwh did the work for Python 2.4 threading, he might be the best person to judge this patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1088832&group_id=5470 From noreply at sourceforge.net Mon Dec 11 05:48:02 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 10 Dec 2006 20:48:02 -0800 Subject: [Patches] [ python-Patches-1090482 ] Patch for bug 999042. Message-ID: Patches item #1090482, was opened at 2004-12-23 19:07 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1090482&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: Python 2.5 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Dariusz Suchojad (dsuch) Assigned to: Neil Schemenauer (nascheme) Summary: Patch for bug 999042. Initial Comment: Hello, I believe this one-line patch fixes [999042] "Compiler module doesn't handle global statement correctly" bug. Without setting the 'optimized' flag compiler.pycodegen.CodeGenerator._nameOp always emits STORE_NAME and not STORE_GLOBAL opcode. After applying test_compiler passes fine. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-11 05:48 Message: Logged In: YES user_id=21627 Originator: NO This patch is wrong. STORE_GLOBAL ought to be used even if optimization is turned off; faking it to be True may have undesirable side effects. The builtin compiler solves this problem by putting the GLOBAL_EXPLICIT scope to a name. The compiler package should do the same. Rejecting this patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1090482&group_id=5470 From noreply at sourceforge.net Mon Dec 11 06:02:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 10 Dec 2006 21:02:37 -0800 Subject: [Patches] [ python-Patches-1093468 ] socket leak in SocketServer Message-ID: Patches item #1093468, was opened at 2004-12-30 21:57 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1093468&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Works For Me Priority: 5 Private: No Submitted By: Shannon -jj Behrens (jjinux) Assigned to: Nobody/Anonymous (nobody) Summary: socket leak in SocketServer Initial Comment: Found in Python 2.3.4 but still present in 2.5. FreeBSD 4.9 Summary: Fixed a socket leak in SocketServer.StreamRequestHandler that happens during certain exceptions. Longer version: SocketServer.StreamRequestHandler.setup sets up self.rfile and self.wfile. The parent class, SocketServer.BaseRequestHandler.__init__ does not call self.finish() if there is an exception thrown in self.handle(). Hence, such an exception leads to a leaked socket, because SocketServer.StreamRequestHandler.finish never gets called. This is a one line patch that fixes that. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-11 06:02 Message: Logged In: YES user_id=21627 Originator: NO I still cannot reproduce the problem, and close this report. When I do the netstat, I don't see any sockets "open", instead, I see many sockets TIME_WAIT. Even after I apply the patch, this behaviour doesn't change: the sockets are still TIME_WAIT. That's actually because the operating system follows the RFCs here: it should keep entries in the socket table just in case additional packets are received after the connection shutdown. I also cannot reproduce the problem with the browser. My browser behaves just fine when I break do_GET. ---------------------------------------------------------------------- Comment By: Shannon -jj Behrens (jjinux) Date: 2005-02-24 23:23 Message: Logged In: YES user_id=30164 I guess the reason this bothers me so much is that if you have a nasty exception in your subclass of SimpleHTTPServer, because the server doesn't close the connection to the browser, the browser will just sit and wait instead of telling the user something bad happened. My patch fixes that. ---------------------------------------------------------------------- Comment By: Shannon -jj Behrens (jjinux) Date: 2005-02-24 23:10 Message: Logged In: YES user_id=30164 Edit SimpleHTTPServer.do_GET: Add "raise ValueError" at the top of the method. python SimpleHTTPServer.py Now, using a browser, go to http://localhost:8000, and reload the page many times. netstat -A inet --tcp | grep 8000 You will see many sockets open. They do go away (so I was wrong), but it may take a minute or two (literally). Just as a proof of concept, I tested to see whether I could bring the box down via this. Fortunately, it doesn't work :-/ It hovers at around 9000 open sockets, with the GC fighting to keep them down. The GC will occassionally take 10-15 seconds trying to clear everything up. Despite the fact the my patch is very simple, I think I'm going to lose this argument :-/ ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-02-24 21:14 Message: Logged In: YES user_id=21627 jjinux, I find it hard to believe that the sockets don't get closed. Normally, with an exception, local variables survive in the traceback. Therefore, SocketServer.py sets sys.exc_traceback to None. It still might be that the SocketServer is part of a cyclic object, in which case the socket won't be closed until the GC actually runs. However, I believe the intention is that the SocketServer normally isn't cyclic. Can you provide a test case showing the problem? ---------------------------------------------------------------------- Comment By: Shannon -jj Behrens (jjinux) Date: 2005-01-18 20:59 Message: Logged In: YES user_id=30164 Unfortunately, that is not the case according to my testing. I can get it to leak sockets and never reclaim them. :-/ ---------------------------------------------------------------------- Comment By: Irmen de Jong (irmen) Date: 2005-01-16 16:55 Message: Logged In: YES user_id=129426 Looks harmless enough but aren't the sockets just garbage collected once the request has been handled? Because the request handler class is instantiated for each request and so it will be removed once done, taking the sockets with it, right? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1093468&group_id=5470 From noreply at sourceforge.net Mon Dec 11 11:10:00 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 11 Dec 2006 02:10:00 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 14:29 Message generated for change (Comment added) made by mkam You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) Assigned to: Thomas Heller (theller) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- >Comment By: Martin Kammerhofer (mkam) Date: 2006-12-11 11:10 Message: Logged In: YES user_id=1656067 Originator: YES Hm, I did not know that OpenBSD is still using two version numbers for shared library. (I conclude that from the "libc.so.39.0" in the previous followup. Btw FreeBSD has used a MAJOR.MINOR[.DEWEY] scheme during the ancient days of the aout executable format.) Unfortunately my freebsd patch has the assumption of a single version number built in; more specifically the cmp(* map(lambda x: int(x.split('.')[-1]), (a, b))) is supposed to sort based an the last dot separated field. I guess that OpenBSD system does not have another libc, at least none with a minor > 0. ;-) Thomas, can you mail me the output of "ldconfig -r"? I will refine the patch then, doing a more general sort algorithm; i.e. sort by all trailing /(\.\d+)+/ fields. Said output from NetBSD welcome too. DragonflyBSD should be no problem since it is a fork of FreeBSD 4.8, but what looks its sys.platform like? ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-08 21:32 Message: Logged In: YES user_id=11105 Originator: NO I have tested the patch on FreeBSD 6.0 and (after extending the check to test for sys.platform.startswith("openbsd")) on OpenBSD 3.9 and it works fine. find_library("c") now returns libc.so.6 on FreeBSD 6.0, and libc.so.39.0 in OpenBSD 3.9, while it returned 'None' before on both machines. ---------------------------------------------------------------------- Comment By: David Remahl (chmod007) Date: 2006-12-08 08:50 Message: Logged In: YES user_id=2135 Originator: NO # Does this work (without the gcc fallback) on other *BSD systems too? I don't know, but it doesn't work on Darwin (which already has a custom method through macholib). ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-07 22:11 Message: Logged In: YES user_id=11105 Originator: NO Will do (although I would appreciate review from others too; I'm not exactly a BSD expert). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-07 20:15 Message: Logged In: YES user_id=21627 Originator: NO Thomas, can you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Mon Dec 11 15:15:20 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 11 Dec 2006 06:15:20 -0800 Subject: [Patches] [ python-Patches-1609407 ] traceback on exit if syslog handler fails to initialize Message-ID: Patches item #1609407, was opened at 2006-12-05 16:43 Message generated for change (Comment added) made by vsajip You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609407&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Jeremy Katz (katzj) Assigned to: Vinay Sajip (vsajip) Summary: traceback on exit if syslog handler fails to initialize Initial Comment: Syslog loggers can fail to initialize if the syslogd has crashed; this then leads to a traceback on exit as self.unixsocket isn't set. Attached patch changes around the order to avoid this. Related to http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=206474 ---------------------------------------------------------------------- >Comment By: Vinay Sajip (vsajip) Date: 2006-12-11 14:15 Message: Logged In: YES user_id=308438 Originator: NO Updated in SVN for trunk and release24-maint. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-09 08:56 Message: Logged In: YES user_id=849994 Originator: NO Assigning to Vinay. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1609407&group_id=5470 From imd at jomiro.pl Mon Dec 11 17:20:11 2006 From: imd at jomiro.pl (Owen Agnes) Date: Mon, 11 Dec 2006 17:20:11 +0100 Subject: [Patches] To use this portlet in a production environment, you must add more thorough error-handling tasks. Message-ID: <000c01c71d40$9cb41af0$4f1e5154@xfzeq> As provisioning for errors, set the error message returned. As I create blocks of code in the Java Studio Creator IDE, it creates JavaBeans components for me. On the other hand, Google Maps is fast and easy to use because it supports AJAX-based drag-and-zoom. This is not a heavily attended conference. Did you know that jMaki is integrated with NetBeans and works with Phobos, JSF, and JSP? Other mashups offer product listings, ratings, auction prices, and so forth by combining catalog data from Amazon with auction data from eBay. This is not a heavily attended conference. To exclude a column you need to use the RowSet's insertableColumns property. There are several products on the market already that do this efficiently; and it is moving from more specialized event driven solutions to the main stream web servers too. For your application, evaluate all the online mapping-service APIs and choose the one that best suits you. Web applications are moving away from the page metaphor to an interactive application model. This month we have a great collection of tools and resources, especially selected for students and beginning developers. Typically this means that only part of the page is updated. If you reside outside the US, please select a country to inquire about products delivered in your country. Only once it gets data will it actually write data into the response for the HTTP GET. and, I understand its completely open source. The following graphic shows how these technologies work together toupdate a piece of a page with new data from the server. In some cases, you may have to obtain the key to the insert operation through some vendor-specific means. My personal favorite is the interview with Sang Shin, Java Technology Architect and Evangelist. Significantly, the server does not send back an entire page, as is the case for conventional, "click, wait, and refresh" web applications. Form component family and renderer type. Garrett said that people often ask him what technologies constitute Ajax. Finally, call commitChanges to commit the changes to the data source. To get them, connect to the Internet, launch Java Studio Creator, and go to the Update Center. I'll call this a Page Bean because its scope is limited to the one page I am on, but it conforms to the JavaBeans standard. Although the experts will try to answer as many questions as feasible, there is no guarantee that all questions will be answered. Learn from technology experts how to simplify application development and improve productivity with the latest emerging Java technologies. With this class, I can more easily produce my objects since each object is so similar in scope. The session will look at the way some of the most commonly used Ajax frameworks integrate with Spring. In his talk, Garrett, cited some of the characteristics that distinguish the new generation of web applications. So now I'm diving into a series of Java technology reference books and SDN articles to get a firsthand understanding of what makes Java technology tick. By using the parent's Maps directly, the developer can avoid the usual paradigm of convenience accessor and mutator methods that are typically defined in a custom JavaServer Faces component. You can incorporate these technologies into a web application in a variety of ways. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20061211/36533fdb/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 9066 bytes Desc: not available Url : http://mail.python.org/pipermail/patches/attachments/20061211/36533fdb/attachment.gif From noreply at sourceforge.net Mon Dec 11 18:22:14 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 11 Dec 2006 09:22:14 -0800 Subject: [Patches] [ python-Patches-1613352 ] encoding directive -- more examples Message-ID: Patches item #1613352, was opened at 2006-12-11 12:22 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613352&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jim Jewett (jimjjewett) Assigned to: Nobody/Anonymous (nobody) Summary: encoding directive -- more examples Initial Comment: As pointed out in http://groups.google.com/group/comp.lang.python/browse_frm/thread/c62220ff0f4cb30a many people read PEP 263 as having stricter requirements -- matching the format shown in the "defining the encoding" example. Make it clear that the regex is the only requirement, with additional examples and a slight wording change. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613352&group_id=5470 From noreply at sourceforge.net Mon Dec 11 18:29:45 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 11 Dec 2006 09:29:45 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 15:53 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 12:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 20:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Mon Dec 11 22:16:56 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 11 Dec 2006 13:16:56 -0800 Subject: [Patches] [ python-Patches-1613500 ] Write mode option for fileinput module. Message-ID: Patches item #1613500, was opened at 2006-12-11 21:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Anthony Roy (antroy) Assigned to: Nobody/Anonymous (nobody) Summary: Write mode option for fileinput module. Initial Comment: The current implementation of the fileinput module allows an optional 'mode' keyword argumant to the FileInput initializer and the input() convenience function. The attached patch provides a similar write_mode keyword argument with which to specify the write mode when the 'inplace' option is used. Included in the patch file are: 1) Changes to the fileinput module to accomodate the optional write mode option 2) Additions to the fileinput tex documentation. 3) A refactored test module, where the tests have been reorganised into TestCase subclasses, and new tests added for the new functionality. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613500&group_id=5470 From tutst at renescent.force9.co.uk Tue Dec 12 04:16:57 2006 From: tutst at renescent.force9.co.uk (paprika) Date: Tue, 12 Dec 2006 04:16:57 +0100 Subject: [Patches] Connetica SoftwareEvova Technology has taken on all software developed byConnetica Software. Message-ID: <001f01c71d9c$77c150d0$6837d5d5@ypnq> The new software, based on open standards, is part of the IBM Build to Manage Toolkit for Problem Determination. Such references do not imply that IBM intends to announce such products, programs or services in your country. This has got to be the simplest way to create irregular shaped windows! With this release, AllMusic has folded in the contents of their classical music Web site, AllClassical. You can also elect to receive a weekly e-mail newsletter listing "noteworthy new releases", with links to on-site reviews provided. They can also browse the music preferences and contributed content of "neighbors" who share similar downloading patterns. Kettlebell training has exploded in that time and with the advent of certified instructors, much new information and training expertise has appeared on the horizon. Once a user selects a news source and a particular video clip, for example, the spoken words in this foreign language file are converted into written words. - Now press the kettlebell towards the sky, rotating your hand towards you so that you end up in what feels like a one-armed bench press position. Today, a glitch in one IT component can trigger dozens of other errors, causing a cascade of problems. Saint Nicolas : nos enfants l'adorent ! The software you are using to access our iobloggo. It is up to you to take precautions to ensure that whatever you select for your use is free of such items as viruses, worms, trojan horses and other items of a destructive nature. Mr Kumar added "IBM's expanding partner networks, and wide range of tested application development solutions have contributed to IBM's consistent leadership in the application development market". - Curl it upwards so that your lower arm is now pointing upward, holding the kettlebell. Caveat: Be VERY careful with this exercise because for the majority of it the kettlebell is above your head while you are balancing it from a precarious position. The funny thing is that if I had just waited a week before tackling the bigger kettlebell, I probably would have been fine. Both of these kits are an excellent way to get into working out with kettlebells. My wife has gone to Gold's with a friend on a guest pass a few times and really likes their "Body Pump" classes, so for our anniversary I thought I'd go ahead and get us a couple of memberships. Consult your local IBM business contact for information regarding the products, programs and services which may be available to you. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/patches/attachments/20061212/ac43e3f1/attachment.html -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/gif Size: 10176 bytes Desc: not available Url : http://mail.python.org/pipermail/patches/attachments/20061212/ac43e3f1/attachment.gif From noreply at sourceforge.net Tue Dec 12 12:28:14 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 12 Dec 2006 03:28:14 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 14:29 Message generated for change (Comment added) made by mkam You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) Assigned to: Thomas Heller (theller) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- >Comment By: Martin Kammerhofer (mkam) Date: 2006-12-12 12:28 Message: Logged In: YES user_id=1656067 Originator: YES Here is the revised patch. Tested on a (virtual) OpenBSD 3.9 machine, FreeBSD 4.11, FreeBSD 6.2 and DragonFlyBSD 1.6. Does not make assumptions on how many version numbers are appended to a library name any more. Even mixed length names (e.g. libfoo.so.8.9 vs. libfoo.so.10) compare in a meaningful way. (BTW: I also tried NetBSD 2.0.2, but its ldconfig is to different.) File Added: ctypes-util.py.patch ---------------------------------------------------------------------- Comment By: Martin Kammerhofer (mkam) Date: 2006-12-11 11:10 Message: Logged In: YES user_id=1656067 Originator: YES Hm, I did not know that OpenBSD is still using two version numbers for shared library. (I conclude that from the "libc.so.39.0" in the previous followup. Btw FreeBSD has used a MAJOR.MINOR[.DEWEY] scheme during the ancient days of the aout executable format.) Unfortunately my freebsd patch has the assumption of a single version number built in; more specifically the cmp(* map(lambda x: int(x.split('.')[-1]), (a, b))) is supposed to sort based an the last dot separated field. I guess that OpenBSD system does not have another libc, at least none with a minor > 0. ;-) Thomas, can you mail me the output of "ldconfig -r"? I will refine the patch then, doing a more general sort algorithm; i.e. sort by all trailing /(\.\d+)+/ fields. Said output from NetBSD welcome too. DragonflyBSD should be no problem since it is a fork of FreeBSD 4.8, but what looks its sys.platform like? ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-08 21:32 Message: Logged In: YES user_id=11105 Originator: NO I have tested the patch on FreeBSD 6.0 and (after extending the check to test for sys.platform.startswith("openbsd")) on OpenBSD 3.9 and it works fine. find_library("c") now returns libc.so.6 on FreeBSD 6.0, and libc.so.39.0 in OpenBSD 3.9, while it returned 'None' before on both machines. ---------------------------------------------------------------------- Comment By: David Remahl (chmod007) Date: 2006-12-08 08:50 Message: Logged In: YES user_id=2135 Originator: NO # Does this work (without the gcc fallback) on other *BSD systems too? I don't know, but it doesn't work on Darwin (which already has a custom method through macholib). ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-07 22:11 Message: Logged In: YES user_id=11105 Originator: NO Will do (although I would appreciate review from others too; I'm not exactly a BSD expert). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-07 20:15 Message: Logged In: YES user_id=21627 Originator: NO Thomas, can you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Wed Dec 13 19:10:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 13 Dec 2006 10:10:43 -0800 Subject: [Patches] [ python-Patches-1615158 ] POSIX capabilities support Message-ID: Patches item #1615158, was opened at 2006-12-13 18:10 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matt Kern (gj0aqzda) Assigned to: Nobody/Anonymous (nobody) Summary: POSIX capabilities support Initial Comment: Attached is a patch which adds POSIX capabilities support. The following API functions are supported: * cap_clear * cap_copy_ext * cap_dup * cap_from_text * cap_get_flag * cap_get_proc * cap_init * cap_set_flag * cap_set_proc * cap_size * cap_to_text The following API function is supported, but is broken with certain versions of libcap (I am running debian testing's libcap1, version 1.10-14, which has an issue; I have reported this upstream): * cap_copy_int The following API functions are in there as stubs, but currently are not compiled. I need access to a machine to test these. I will probably add autoconf tests for availability of these functions in due course: * cap_get_fd * cap_get_file * cap_set_fd * cap_set_file The patch includes diffs to configure. My autoconf is however at a different revision to that used on the python trunk. You may want to re-autoconf configure.in. I've added a few API tests to test_posix.py. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 From noreply at sourceforge.net Wed Dec 13 19:12:56 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 13 Dec 2006 10:12:56 -0800 Subject: [Patches] [ python-Patches-1615158 ] POSIX capabilities support Message-ID: Patches item #1615158, was opened at 2006-12-13 18:10 Message generated for change (Comment added) made by gj0aqzda You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matt Kern (gj0aqzda) Assigned to: Nobody/Anonymous (nobody) Summary: POSIX capabilities support Initial Comment: Attached is a patch which adds POSIX capabilities support. The following API functions are supported: * cap_clear * cap_copy_ext * cap_dup * cap_from_text * cap_get_flag * cap_get_proc * cap_init * cap_set_flag * cap_set_proc * cap_size * cap_to_text The following API function is supported, but is broken with certain versions of libcap (I am running debian testing's libcap1, version 1.10-14, which has an issue; I have reported this upstream): * cap_copy_int The following API functions are in there as stubs, but currently are not compiled. I need access to a machine to test these. I will probably add autoconf tests for availability of these functions in due course: * cap_get_fd * cap_get_file * cap_set_fd * cap_set_file The patch includes diffs to configure. My autoconf is however at a different revision to that used on the python trunk. You may want to re-autoconf configure.in. I've added a few API tests to test_posix.py. ---------------------------------------------------------------------- >Comment By: Matt Kern (gj0aqzda) Date: 2006-12-13 18:12 Message: Logged In: YES user_id=1667774 Originator: YES I should further add that I have implemented the following API calls as methods of the new CapabilityState object in addition to the standard functions: * cap_clear * cap_copy_ext * cap_dup * cap_get_flag * cap_set_flag * cap_set_proc * cap_size * cap_to_text ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 From noreply at sourceforge.net Thu Dec 14 14:08:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 14 Dec 2006 05:08:32 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 14:08 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Thu Dec 14 17:25:38 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 14 Dec 2006 08:25:38 -0800 Subject: [Patches] [ python-Patches-1615868 ] BZ2File.seek() fails for large files Message-ID: Patches item #1615868, was opened at 2006-12-14 17:25 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615868&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: BZ2File.seek() fails for large files Initial Comment: The bytesread variable in BZ2File.seek() is defined as int instead of size_t making it impossible to seek beyond 2GB distances. Affected are all Python versions. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615868&group_id=5470 From noreply at sourceforge.net Thu Dec 14 17:45:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 14 Dec 2006 08:45:53 -0800 Subject: [Patches] [ python-Patches-1615868 ] BZ2File.seek() fails for large files Message-ID: Patches item #1615868, was opened at 2006-12-14 17:25 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615868&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: BZ2File.seek() fails for large files Initial Comment: The bytesread variable in BZ2File.seek() is defined as int instead of size_t making it impossible to seek beyond 2GB distances. Affected are all Python versions. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-14 17:45 Message: Logged In: YES user_id=642936 Originator: YES Nah, it must be Py_off_t. File Added: bz2-seek.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615868&group_id=5470 From noreply at sourceforge.net Thu Dec 14 19:51:26 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 14 Dec 2006 10:51:26 -0800 Subject: [Patches] [ python-Patches-1599256 ] mailbox.py: check that os.fsync is available before using it Message-ID: Patches item #1599256, was opened at 2006-11-19 11:12 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1599256&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: David Watson (baikie) >Assigned to: A.M. Kuchling (akuchling) Summary: mailbox.py: check that os.fsync is available before using it Initial Comment: I forgot to do this in patch #1514544. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1599256&group_id=5470 From noreply at sourceforge.net Thu Dec 14 19:59:23 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 14 Dec 2006 10:59:23 -0800 Subject: [Patches] [ python-Patches-1599256 ] mailbox.py: check that os.fsync is available before using it Message-ID: Patches item #1599256, was opened at 2006-11-19 11:12 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1599256&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: David Watson (baikie) Assigned to: A.M. Kuchling (akuchling) Summary: mailbox.py: check that os.fsync is available before using it Initial Comment: I forgot to do this in patch #1514544. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-14 13:59 Message: Logged In: YES user_id=11375 Originator: NO Committed to trunk in rev. 53032 and to 25-maint in rev. 53033. Thanks! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1599256&group_id=5470 From noreply at sourceforge.net Thu Dec 14 20:00:14 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 14 Dec 2006 11:00:14 -0800 Subject: [Patches] [ python-Patches-1603907 ] subprocess: error redirecting i/o from non-console process Message-ID: Patches item #1603907, was opened at 2006-11-27 12:20 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1603907&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Oren Tirosh (orenti) >Assigned to: Peter ?strand (astrand) Summary: subprocess: error redirecting i/o from non-console process Initial Comment: In IDLE, PythonWin or other non-console interactive Python under Windows: >>> from subprocess import * >>> Popen('cmd', stdout=PIPE) Traceback (most recent call last): File "", line 1, in -toplevel- Popen('', stdout=PIPE) File "C:\python24\lib\subprocess.py", line 533, in __init__ (p2cread, p2cwrite, File "C:\python24\lib\subprocess.py", line 593, in _get_handles p2cread = self._make_inheritable(p2cread) File "C:\python24\lib\subprocess.py", line 634, in _make_inheritable DUPLICATE_SAME_ACCESS) TypeError: an integer is required The same command in a console windows is successful. Why it happens: subprocess assumes that GetStdHandle always succeeds but when there is no console it returns None. DuplicateHandle then complains about getting a non-integer. This problem does not happen when redirecting all three standard handles. Solution: Replace None with -1 (INVALID_HANDLE_VALUE) in _make_inheritable. Patch attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1603907&group_id=5470 From noreply at sourceforge.net Fri Dec 15 01:44:46 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 14 Dec 2006 16:44:46 -0800 Subject: [Patches] [ python-Patches-1616125 ] Cached globals+builtins lookup optimization Message-ID: Patches item #1616125, was opened at 2006-12-15 01:44 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrea Griffini (ag6502) Assigned to: Nobody/Anonymous (nobody) Summary: Cached globals+builtins lookup optimization Initial Comment: This patch implements a speed optimization by introducing a timestamp on dictionaries and by caching lookups of constants so that lookup doesn't need to be repeated if the dictionary didn't change. Currently the patch implements the cached lookup only for the LOAD_GLOBAL opcode and stores the cache as two extra member in the code object. I'm going to investigate if LOAD_ATTR in the special case of a module is worth caching too. A cache entry for LOAD_GLOBAL is defined by two timestamps (Py_ssize_t) and a *borrowed* pointer to a PyObject. The use of a borrowed pointer is safe because the value is used only if the dictionaries didn't change, so the result of lookup must still be alive because the reference in the dictionary that returned the cached value it is still in place. The assumptions are: - a LOAD_GLOBAL opcode always looks for the same symbol (i.e. co_names never changes) - the dictionary being searched (globals and builtins) are mostly constant On my system I got a speedup of 8%-10% on a real program (a tcp/ip server that handles an in-memory database of python objects) on the checkpoint load + rollforward steps (a few millions LOAD_GLOBAL operations in total) and the varying percentage depends on which compiler options are used to build python. On another little test program that calls a function def f(x): return math.sin(x) + math.cos(x) + abs(x) one million times the speedup with the default makefile is about 7%. To enable the patch the makefile should define the CACHED_LOOKUPS symbol. The change to dictobject also exports the timestamp, at the python level; other changes are instead invisible at the python level. I'm not sure at all exporting dict.timestamp() is a good idea however... Note that after applying the patch you may find an error in userdict if you simply use "make" (there's probably some dependency problem somewhere). After make clean + make the test suite runs fine on my system. I'm interested in hearing if also other platforms do actually see a real performance improvement. Andrea ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 From noreply at sourceforge.net Fri Dec 15 14:10:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 15 Dec 2006 05:10:22 -0800 Subject: [Patches] [ python-Patches-1613500 ] Write mode option for fileinput module. Message-ID: Patches item #1613500, was opened at 2006-12-11 22:16 Message generated for change (Comment added) made by mkam You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Anthony Roy (antroy) Assigned to: Nobody/Anonymous (nobody) Summary: Write mode option for fileinput module. Initial Comment: The current implementation of the fileinput module allows an optional 'mode' keyword argumant to the FileInput initializer and the input() convenience function. The attached patch provides a similar write_mode keyword argument with which to specify the write mode when the 'inplace' option is used. Included in the patch file are: 1) Changes to the fileinput module to accomodate the optional write mode option 2) Additions to the fileinput tex documentation. 3) A refactored test module, where the tests have been reorganised into TestCase subclasses, and new tests added for the new functionality. ---------------------------------------------------------------------- Comment By: Martin Kammerhofer (mkam) Date: 2006-12-15 14:10 Message: Logged In: YES user_id=1656067 Originator: NO The restriction that you cannot use inplace and openhook together seems somewhat arbitrary to me. Have you considered lifting it? From looking at the source it seems not too hard to do. There are uses for e.g. inplace editing compressed files. This would fit nicely with your patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613500&group_id=5470 From noreply at sourceforge.net Sat Dec 16 12:01:24 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 03:01:24 -0800 Subject: [Patches] [ python-Patches-1616125 ] Cached globals+builtins lookup optimization Message-ID: Patches item #1616125, was opened at 2006-12-15 01:44 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrea Griffini (ag6502) Assigned to: Nobody/Anonymous (nobody) Summary: Cached globals+builtins lookup optimization Initial Comment: This patch implements a speed optimization by introducing a timestamp on dictionaries and by caching lookups of constants so that lookup doesn't need to be repeated if the dictionary didn't change. Currently the patch implements the cached lookup only for the LOAD_GLOBAL opcode and stores the cache as two extra member in the code object. I'm going to investigate if LOAD_ATTR in the special case of a module is worth caching too. A cache entry for LOAD_GLOBAL is defined by two timestamps (Py_ssize_t) and a *borrowed* pointer to a PyObject. The use of a borrowed pointer is safe because the value is used only if the dictionaries didn't change, so the result of lookup must still be alive because the reference in the dictionary that returned the cached value it is still in place. The assumptions are: - a LOAD_GLOBAL opcode always looks for the same symbol (i.e. co_names never changes) - the dictionary being searched (globals and builtins) are mostly constant On my system I got a speedup of 8%-10% on a real program (a tcp/ip server that handles an in-memory database of python objects) on the checkpoint load + rollforward steps (a few millions LOAD_GLOBAL operations in total) and the varying percentage depends on which compiler options are used to build python. On another little test program that calls a function def f(x): return math.sin(x) + math.cos(x) + abs(x) one million times the speedup with the default makefile is about 7%. To enable the patch the makefile should define the CACHED_LOOKUPS symbol. The change to dictobject also exports the timestamp, at the python level; other changes are instead invisible at the python level. I'm not sure at all exporting dict.timestamp() is a good idea however... Note that after applying the patch you may find an error in userdict if you simply use "make" (there's probably some dependency problem somewhere). After make clean + make the test suite runs fine on my system. I'm interested in hearing if also other platforms do actually see a real performance improvement. Andrea ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 12:01 Message: Logged In: YES user_id=21627 Originator: NO How does that deal with timestamp wraparound? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 From noreply at sourceforge.net Sat Dec 16 13:28:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 04:28:53 -0800 Subject: [Patches] [ python-Patches-1615158 ] POSIX capabilities support Message-ID: Patches item #1615158, was opened at 2006-12-13 19:10 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matt Kern (gj0aqzda) Assigned to: Nobody/Anonymous (nobody) Summary: POSIX capabilities support Initial Comment: Attached is a patch which adds POSIX capabilities support. The following API functions are supported: * cap_clear * cap_copy_ext * cap_dup * cap_from_text * cap_get_flag * cap_get_proc * cap_init * cap_set_flag * cap_set_proc * cap_size * cap_to_text The following API function is supported, but is broken with certain versions of libcap (I am running debian testing's libcap1, version 1.10-14, which has an issue; I have reported this upstream): * cap_copy_int The following API functions are in there as stubs, but currently are not compiled. I need access to a machine to test these. I will probably add autoconf tests for availability of these functions in due course: * cap_get_fd * cap_get_file * cap_set_fd * cap_set_file The patch includes diffs to configure. My autoconf is however at a different revision to that used on the python trunk. You may want to re-autoconf configure.in. I've added a few API tests to test_posix.py. ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 13:28 Message: Logged In: YES user_id=21627 Originator: NO Can you please provide documentation changes as well? ---------------------------------------------------------------------- Comment By: Matt Kern (gj0aqzda) Date: 2006-12-13 19:12 Message: Logged In: YES user_id=1667774 Originator: YES I should further add that I have implemented the following API calls as methods of the new CapabilityState object in addition to the standard functions: * cap_clear * cap_copy_ext * cap_dup * cap_get_flag * cap_set_flag * cap_set_proc * cap_size * cap_to_text ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 From noreply at sourceforge.net Sat Dec 16 14:25:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 05:25:25 -0800 Subject: [Patches] [ python-Patches-1615158 ] POSIX capabilities support Message-ID: Patches item #1615158, was opened at 2006-12-13 18:10 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matt Kern (gj0aqzda) Assigned to: Nobody/Anonymous (nobody) Summary: POSIX capabilities support Initial Comment: Attached is a patch which adds POSIX capabilities support. The following API functions are supported: * cap_clear * cap_copy_ext * cap_dup * cap_from_text * cap_get_flag * cap_get_proc * cap_init * cap_set_flag * cap_set_proc * cap_size * cap_to_text The following API function is supported, but is broken with certain versions of libcap (I am running debian testing's libcap1, version 1.10-14, which has an issue; I have reported this upstream): * cap_copy_int The following API functions are in there as stubs, but currently are not compiled. I need access to a machine to test these. I will probably add autoconf tests for availability of these functions in due course: * cap_get_fd * cap_get_file * cap_set_fd * cap_set_file The patch includes diffs to configure. My autoconf is however at a different revision to that used on the python trunk. You may want to re-autoconf configure.in. I've added a few API tests to test_posix.py. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-16 13:25 Message: Logged In: YES user_id=849994 Originator: NO (If you don't want to write LaTeX, it's enough to write the docs in plaintext, there are a few volunteers who will convert it appropriately.) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 12:28 Message: Logged In: YES user_id=21627 Originator: NO Can you please provide documentation changes as well? ---------------------------------------------------------------------- Comment By: Matt Kern (gj0aqzda) Date: 2006-12-13 18:12 Message: Logged In: YES user_id=1667774 Originator: YES I should further add that I have implemented the following API calls as methods of the new CapabilityState object in addition to the standard functions: * cap_clear * cap_copy_ext * cap_dup * cap_get_flag * cap_set_flag * cap_set_proc * cap_size * cap_to_text ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 From noreply at sourceforge.net Sat Dec 16 15:24:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 06:24:07 -0800 Subject: [Patches] [ python-Patches-1616979 ] cp720 encoding map Message-ID: Patches item #1616979, was opened at 2006-12-16 16:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616979&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Alexander Belchenko (bialix) Assigned to: Nobody/Anonymous (nobody) Summary: cp720 encoding map Initial Comment: I'm working on Bazaar (bzr) VCS. One of our user report about bug that occurs because of his Windows XP machine use cp720 codepage for DOS console. cp720 is OEM Arabic codepage. Python standard library does not have encoding map for this encoding so I create corresponding one. Attached patch provide cp720.py file for encodings package and mention this encoding in documentation. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616979&group_id=5470 From noreply at sourceforge.net Sat Dec 16 18:13:38 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 09:13:38 -0800 Subject: [Patches] [ python-Patches-1616125 ] Cached globals+builtins lookup optimization Message-ID: Patches item #1616125, was opened at 2006-12-15 01:44 Message generated for change (Comment added) made by ag6502 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrea Griffini (ag6502) Assigned to: Nobody/Anonymous (nobody) Summary: Cached globals+builtins lookup optimization Initial Comment: This patch implements a speed optimization by introducing a timestamp on dictionaries and by caching lookups of constants so that lookup doesn't need to be repeated if the dictionary didn't change. Currently the patch implements the cached lookup only for the LOAD_GLOBAL opcode and stores the cache as two extra member in the code object. I'm going to investigate if LOAD_ATTR in the special case of a module is worth caching too. A cache entry for LOAD_GLOBAL is defined by two timestamps (Py_ssize_t) and a *borrowed* pointer to a PyObject. The use of a borrowed pointer is safe because the value is used only if the dictionaries didn't change, so the result of lookup must still be alive because the reference in the dictionary that returned the cached value it is still in place. The assumptions are: - a LOAD_GLOBAL opcode always looks for the same symbol (i.e. co_names never changes) - the dictionary being searched (globals and builtins) are mostly constant On my system I got a speedup of 8%-10% on a real program (a tcp/ip server that handles an in-memory database of python objects) on the checkpoint load + rollforward steps (a few millions LOAD_GLOBAL operations in total) and the varying percentage depends on which compiler options are used to build python. On another little test program that calls a function def f(x): return math.sin(x) + math.cos(x) + abs(x) one million times the speedup with the default makefile is about 7%. To enable the patch the makefile should define the CACHED_LOOKUPS symbol. The change to dictobject also exports the timestamp, at the python level; other changes are instead invisible at the python level. I'm not sure at all exporting dict.timestamp() is a good idea however... Note that after applying the patch you may find an error in userdict if you simply use "make" (there's probably some dependency problem somewhere). After make clean + make the test suite runs fine on my system. I'm interested in hearing if also other platforms do actually see a real performance improvement. Andrea ---------------------------------------------------------------------- >Comment By: Andrea Griffini (ag6502) Date: 2006-12-16 18:13 Message: Logged In: YES user_id=1668946 Originator: YES I am experimenting with long long timestamps (based on "unsigned long long" if available or explicitly on two "unsigned long" otherwise). An alternative to slowing down lookups by using 64 bits on 32-bit computers is just using a single 32-bit counter and trigger a "cache flush" when the timestamp rolls over by visiting all live dictionaries and code objects. This would be IMO a *very* infrequent operation (for example on my P4/2.4GHz just doing nothing else but dict updates getting to 2**32 updates would require 14 minutes). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 12:01 Message: Logged In: YES user_id=21627 Originator: NO How does that deal with timestamp wraparound? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 From noreply at sourceforge.net Sat Dec 16 23:35:18 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 14:35:18 -0800 Subject: [Patches] [ python-Patches-1362975 ] CodeContext - Improved text indentation Message-ID: Patches item #1362975, was opened at 2005-11-21 20:06 Message generated for change (Comment added) made by taleinat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Closed Resolution: Accepted Priority: 3 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Kurt B. Kaiser (kbk) Summary: CodeContext - Improved text indentation Initial Comment: This is a second patch for the text indentation - the first one was sent by mail and patched directly by KBK. I've touched up the indentation code to be cleaner and more generic. There's no longer need for a pad Frame, the padding is done by the Label widget. More IDLE patches coming - stay tuned ;) ---------------------------------------------------------------------- >Comment By: Tal Einat (taleinat) Date: 2006-12-17 00:35 Message: Logged In: YES user_id=1330769 Originator: YES Revised excessively verbose comments, removed overly defensive try/except blocks, some minor comment and code cleanup. File Added: CodeContext.061217.patch ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-22 16:30 Message: Logged In: YES user_id=1330769 Originator: YES Woohoo! :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-22 10:51 Message: Logged In: YES user_id=21627 Originator: NO Thanks for the patch. I can't find anything wrong anymore, and agree that the restructuring is better, so I have committed it as r52821. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-20 21:06 Message: Logged In: YES user_id=1330769 Originator: YES Seems like a bug in EditorWindow.py to me - shouldn't edtiwin.vbar (the scrollbar) be created inside editwin.text_frame instead of editwin.top? Seems to me that this is the reason text_frame is there in the first place - to encapsulate the Text widget and the scrollbar that accompanies it. Funny that I hadn't noticed the odd placement of the scrollbar... Thanks for the thorough review! I've submitted a revised patch, which fixes the bug in EditorWindow.py. I've also wrapped all widget attribute lookups in try/except blocks to avoid weird Tk behavior. This way future changes in Tk's inner workings will cause the text to be misaligned, instead of crashing IDLE. Lastly, I've added verbose comments where appropriate. Testing on current SVN trunk version of IDLE with only these changes shows no problems, and it works smoothly with my development version of IDLE as well. Let's finally get this piece of code behind us :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-19 21:18 Message: Logged In: YES user_id=21627 Originator: NO The patch is incorrect: without the patch, the scrollbar for the text window ends below the context window. With the patch, the scrollbar is on the side of the context window also. Tentatively rejecting the patch; if you revise it, please add some comments explaining all the computations. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2005-11-23 11:04 Message: Logged In: YES user_id=1330769 This patch is an improvement for two reasons: 1. One less TK widget is used 2. Replaces literal values with code that fetches the appropriate values from the editor window's widgets I'll admit it's not that big a deal. If it's really a lot of work, I'll leave it up to you to decide whether it's worth it. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-11-23 03:25 Message: Logged In: YES user_id=149084 At first glance the new code seems harder to understand and is longer. What is the advantage of going through the effort to apply, test, check in, and properly document the change? p.s. use Expand=False ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 From noreply at sourceforge.net Sat Dec 16 23:57:57 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 14:57:57 -0800 Subject: [Patches] [ python-Patches-1616125 ] Cached globals+builtins lookup optimization Message-ID: Patches item #1616125, was opened at 2006-12-15 01:44 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrea Griffini (ag6502) Assigned to: Nobody/Anonymous (nobody) Summary: Cached globals+builtins lookup optimization Initial Comment: This patch implements a speed optimization by introducing a timestamp on dictionaries and by caching lookups of constants so that lookup doesn't need to be repeated if the dictionary didn't change. Currently the patch implements the cached lookup only for the LOAD_GLOBAL opcode and stores the cache as two extra member in the code object. I'm going to investigate if LOAD_ATTR in the special case of a module is worth caching too. A cache entry for LOAD_GLOBAL is defined by two timestamps (Py_ssize_t) and a *borrowed* pointer to a PyObject. The use of a borrowed pointer is safe because the value is used only if the dictionaries didn't change, so the result of lookup must still be alive because the reference in the dictionary that returned the cached value it is still in place. The assumptions are: - a LOAD_GLOBAL opcode always looks for the same symbol (i.e. co_names never changes) - the dictionary being searched (globals and builtins) are mostly constant On my system I got a speedup of 8%-10% on a real program (a tcp/ip server that handles an in-memory database of python objects) on the checkpoint load + rollforward steps (a few millions LOAD_GLOBAL operations in total) and the varying percentage depends on which compiler options are used to build python. On another little test program that calls a function def f(x): return math.sin(x) + math.cos(x) + abs(x) one million times the speedup with the default makefile is about 7%. To enable the patch the makefile should define the CACHED_LOOKUPS symbol. The change to dictobject also exports the timestamp, at the python level; other changes are instead invisible at the python level. I'm not sure at all exporting dict.timestamp() is a good idea however... Note that after applying the patch you may find an error in userdict if you simply use "make" (there's probably some dependency problem somewhere). After make clean + make the test suite runs fine on my system. I'm interested in hearing if also other platforms do actually see a real performance improvement. Andrea ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 23:57 Message: Logged In: YES user_id=21627 Originator: NO I think restricting yourself to 32-bit counters is fine (it's also an issue of memory overhead as you have one of these per access to a global, and per dictionary); you have to deal with the overflow, though. I suggest to use an unsigned value, e.g. size_t. Overflowing should be dealt with through visiting all objects, as you say. It could either happen right when it overflows, or it could get integrated into GC, and only reset when the GC runs. You'd then need to account for those periods when it already has overflowed, but GC hasn't run yet; in these periods, counting should be "on hold". ---------------------------------------------------------------------- Comment By: Andrea Griffini (ag6502) Date: 2006-12-16 18:13 Message: Logged In: YES user_id=1668946 Originator: YES I am experimenting with long long timestamps (based on "unsigned long long" if available or explicitly on two "unsigned long" otherwise). An alternative to slowing down lookups by using 64 bits on 32-bit computers is just using a single 32-bit counter and trigger a "cache flush" when the timestamp rolls over by visiting all live dictionaries and code objects. This would be IMO a *very* infrequent operation (for example on my P4/2.4GHz just doing nothing else but dict updates getting to 2**32 updates would require 14 minutes). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 12:01 Message: Logged In: YES user_id=21627 Originator: NO How does that deal with timestamp wraparound? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 From noreply at sourceforge.net Sun Dec 17 00:01:41 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 15:01:41 -0800 Subject: [Patches] [ python-Patches-1362975 ] CodeContext - Improved text indentation Message-ID: Patches item #1362975, was opened at 2005-11-21 19:06 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Closed Resolution: Accepted Priority: 3 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Kurt B. Kaiser (kbk) Summary: CodeContext - Improved text indentation Initial Comment: This is a second patch for the text indentation - the first one was sent by mail and patched directly by KBK. I've touched up the indentation code to be cleaner and more generic. There's no longer need for a pad Frame, the padding is done by the Label widget. More IDLE patches coming - stay tuned ;) ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-17 00:01 Message: Logged In: YES user_id=21627 Originator: NO Hmm. This patch is already closed. Should the previous checkin be reverted? If this addresses additional issues, please submit it as a new report. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-12-16 23:35 Message: Logged In: YES user_id=1330769 Originator: YES Revised excessively verbose comments, removed overly defensive try/except blocks, some minor comment and code cleanup. File Added: CodeContext.061217.patch ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-22 15:30 Message: Logged In: YES user_id=1330769 Originator: YES Woohoo! :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-22 09:51 Message: Logged In: YES user_id=21627 Originator: NO Thanks for the patch. I can't find anything wrong anymore, and agree that the restructuring is better, so I have committed it as r52821. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-20 20:06 Message: Logged In: YES user_id=1330769 Originator: YES Seems like a bug in EditorWindow.py to me - shouldn't edtiwin.vbar (the scrollbar) be created inside editwin.text_frame instead of editwin.top? Seems to me that this is the reason text_frame is there in the first place - to encapsulate the Text widget and the scrollbar that accompanies it. Funny that I hadn't noticed the odd placement of the scrollbar... Thanks for the thorough review! I've submitted a revised patch, which fixes the bug in EditorWindow.py. I've also wrapped all widget attribute lookups in try/except blocks to avoid weird Tk behavior. This way future changes in Tk's inner workings will cause the text to be misaligned, instead of crashing IDLE. Lastly, I've added verbose comments where appropriate. Testing on current SVN trunk version of IDLE with only these changes shows no problems, and it works smoothly with my development version of IDLE as well. Let's finally get this piece of code behind us :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-19 20:18 Message: Logged In: YES user_id=21627 Originator: NO The patch is incorrect: without the patch, the scrollbar for the text window ends below the context window. With the patch, the scrollbar is on the side of the context window also. Tentatively rejecting the patch; if you revise it, please add some comments explaining all the computations. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2005-11-23 10:04 Message: Logged In: YES user_id=1330769 This patch is an improvement for two reasons: 1. One less TK widget is used 2. Replaces literal values with code that fetches the appropriate values from the editor window's widgets I'll admit it's not that big a deal. If it's really a lot of work, I'll leave it up to you to decide whether it's worth it. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-11-23 02:25 Message: Logged In: YES user_id=149084 At first glance the new code seems harder to understand and is longer. What is the advantage of going through the effort to apply, test, check in, and properly document the change? p.s. use Expand=False ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 From noreply at sourceforge.net Sun Dec 17 00:40:39 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 16 Dec 2006 15:40:39 -0800 Subject: [Patches] [ python-Patches-1362975 ] CodeContext - Improved text indentation Message-ID: Patches item #1362975, was opened at 2005-11-21 20:06 Message generated for change (Comment added) made by taleinat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Closed Resolution: Accepted Priority: 3 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Kurt B. Kaiser (kbk) Summary: CodeContext - Improved text indentation Initial Comment: This is a second patch for the text indentation - the first one was sent by mail and patched directly by KBK. I've touched up the indentation code to be cleaner and more generic. There's no longer need for a pad Frame, the padding is done by the Label widget. More IDLE patches coming - stay tuned ;) ---------------------------------------------------------------------- >Comment By: Tal Einat (taleinat) Date: 2006-12-17 01:40 Message: Logged In: YES user_id=1330769 Originator: YES Revised excessively verbose comments, removed overly defensive try/except blocks, some minor comment and code cleanup. File Added: CodeContext.061217.patch ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-17 01:01 Message: Logged In: YES user_id=21627 Originator: NO Hmm. This patch is already closed. Should the previous checkin be reverted? If this addresses additional issues, please submit it as a new report. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-12-17 00:35 Message: Logged In: YES user_id=1330769 Originator: YES Revised excessively verbose comments, removed overly defensive try/except blocks, some minor comment and code cleanup. File Added: CodeContext.061217.patch ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-22 16:30 Message: Logged In: YES user_id=1330769 Originator: YES Woohoo! :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-22 10:51 Message: Logged In: YES user_id=21627 Originator: NO Thanks for the patch. I can't find anything wrong anymore, and agree that the restructuring is better, so I have committed it as r52821. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-20 21:06 Message: Logged In: YES user_id=1330769 Originator: YES Seems like a bug in EditorWindow.py to me - shouldn't edtiwin.vbar (the scrollbar) be created inside editwin.text_frame instead of editwin.top? Seems to me that this is the reason text_frame is there in the first place - to encapsulate the Text widget and the scrollbar that accompanies it. Funny that I hadn't noticed the odd placement of the scrollbar... Thanks for the thorough review! I've submitted a revised patch, which fixes the bug in EditorWindow.py. I've also wrapped all widget attribute lookups in try/except blocks to avoid weird Tk behavior. This way future changes in Tk's inner workings will cause the text to be misaligned, instead of crashing IDLE. Lastly, I've added verbose comments where appropriate. Testing on current SVN trunk version of IDLE with only these changes shows no problems, and it works smoothly with my development version of IDLE as well. Let's finally get this piece of code behind us :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-19 21:18 Message: Logged In: YES user_id=21627 Originator: NO The patch is incorrect: without the patch, the scrollbar for the text window ends below the context window. With the patch, the scrollbar is on the side of the context window also. Tentatively rejecting the patch; if you revise it, please add some comments explaining all the computations. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2005-11-23 11:04 Message: Logged In: YES user_id=1330769 This patch is an improvement for two reasons: 1. One less TK widget is used 2. Replaces literal values with code that fetches the appropriate values from the editor window's widgets I'll admit it's not that big a deal. If it's really a lot of work, I'll leave it up to you to decide whether it's worth it. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-11-23 03:25 Message: Logged In: YES user_id=149084 At first glance the new code seems harder to understand and is longer. What is the advantage of going through the effort to apply, test, check in, and properly document the change? p.s. use Expand=False ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 From noreply at sourceforge.net Sun Dec 17 11:00:50 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 02:00:50 -0800 Subject: [Patches] [ python-Patches-1362975 ] CodeContext - Improved text indentation Message-ID: Patches item #1362975, was opened at 2005-11-21 20:06 Message generated for change (Comment added) made by taleinat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: None Status: Closed Resolution: Accepted Priority: 3 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Kurt B. Kaiser (kbk) Summary: CodeContext - Improved text indentation Initial Comment: This is a second patch for the text indentation - the first one was sent by mail and patched directly by KBK. I've touched up the indentation code to be cleaner and more generic. There's no longer need for a pad Frame, the padding is done by the Label widget. More IDLE patches coming - stay tuned ;) ---------------------------------------------------------------------- >Comment By: Tal Einat (taleinat) Date: 2006-12-17 12:00 Message: Logged In: YES user_id=1330769 Originator: YES The changes were at Kurt's request. No new issues are addressed, it's only a revision of the previous changes. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-12-17 01:40 Message: Logged In: YES user_id=1330769 Originator: YES Revised excessively verbose comments, removed overly defensive try/except blocks, some minor comment and code cleanup. File Added: CodeContext.061217.patch ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-17 01:01 Message: Logged In: YES user_id=21627 Originator: NO Hmm. This patch is already closed. Should the previous checkin be reverted? If this addresses additional issues, please submit it as a new report. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-12-17 00:35 Message: Logged In: YES user_id=1330769 Originator: YES Revised excessively verbose comments, removed overly defensive try/except blocks, some minor comment and code cleanup. File Added: CodeContext.061217.patch ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-22 16:30 Message: Logged In: YES user_id=1330769 Originator: YES Woohoo! :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-22 10:51 Message: Logged In: YES user_id=21627 Originator: NO Thanks for the patch. I can't find anything wrong anymore, and agree that the restructuring is better, so I have committed it as r52821. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2006-11-20 21:06 Message: Logged In: YES user_id=1330769 Originator: YES Seems like a bug in EditorWindow.py to me - shouldn't edtiwin.vbar (the scrollbar) be created inside editwin.text_frame instead of editwin.top? Seems to me that this is the reason text_frame is there in the first place - to encapsulate the Text widget and the scrollbar that accompanies it. Funny that I hadn't noticed the odd placement of the scrollbar... Thanks for the thorough review! I've submitted a revised patch, which fixes the bug in EditorWindow.py. I've also wrapped all widget attribute lookups in try/except blocks to avoid weird Tk behavior. This way future changes in Tk's inner workings will cause the text to be misaligned, instead of crashing IDLE. Lastly, I've added verbose comments where appropriate. Testing on current SVN trunk version of IDLE with only these changes shows no problems, and it works smoothly with my development version of IDLE as well. Let's finally get this piece of code behind us :) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-11-19 21:18 Message: Logged In: YES user_id=21627 Originator: NO The patch is incorrect: without the patch, the scrollbar for the text window ends below the context window. With the patch, the scrollbar is on the side of the context window also. Tentatively rejecting the patch; if you revise it, please add some comments explaining all the computations. ---------------------------------------------------------------------- Comment By: Tal Einat (taleinat) Date: 2005-11-23 11:04 Message: Logged In: YES user_id=1330769 This patch is an improvement for two reasons: 1. One less TK widget is used 2. Replaces literal values with code that fetches the appropriate values from the editor window's widgets I'll admit it's not that big a deal. If it's really a lot of work, I'll leave it up to you to decide whether it's worth it. ---------------------------------------------------------------------- Comment By: Kurt B. Kaiser (kbk) Date: 2005-11-23 03:25 Message: Logged In: YES user_id=149084 At first glance the new code seems harder to understand and is longer. What is the advantage of going through the effort to apply, test, check in, and properly document the change? p.s. use Expand=False ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1362975&group_id=5470 From noreply at sourceforge.net Sun Dec 17 17:05:48 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 08:05:48 -0800 Subject: [Patches] [ python-Patches-1617413 ] urllib HTTPS Basic authentication fix Message-ID: Patches item #1617413, was opened at 2006-12-17 11:05 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617413&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Dug Song (dugsong) Assigned to: Nobody/Anonymous (nobody) Summary: urllib HTTPS Basic authentication fix Initial Comment: urllib's support for HTTP Basic authentication via HTTPS is broken, due to an invalid call in URLopener.open_https() to HTTPConnection.putheader() (apparently miscopied from URLopener.open_http(), which gets it right). the code incorrectly calls putheader with a fully formatted header instead of passing a name and value separately, resulting in a badly formatted header ending with a trailing colon. the fix is trivial. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617413&group_id=5470 From noreply at sourceforge.net Sun Dec 17 21:02:10 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 12:02:10 -0800 Subject: [Patches] [ python-Patches-1617496 ] The ability to keep configuration files intact Message-ID: Patches item #1617496, was opened at 2006-12-17 23:02 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617496&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils and setup.py Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Roman Kurakin (sagamore) Assigned to: Nobody/Anonymous (nobody) Summary: The ability to keep configuration files intact Initial Comment: While working on one project I've met a problem that I can't to specify to not tuch my configuration files for python package that uses setup.py technique. To solve this problem I've implemented the following solution that works for me. If this idea worths it, I could made a pacth relative needed version of python. +class smart_install_data(install_data): + def run(self): + # Use revers order for safe removal + for i in range(len(self.data_files)-1, -1, -1): + f = self.data_files[i] + if type(f) is StringType: + continue + if len(f) <= 2: + continue + # Ok, we have additional value, do some magick according it. + if f[2] != "preserve": + # Nope, we do not know it. Just ignore for now. + # Should we scream about it? + continue + # Check a tuple with path to install to and a list of files + dir = convert_path(f[0]) + if not os.path.isabs(dir): + dir = os.path.join(self.install_dir, dir) + elif self.root: + dir = change_root(self.root, dir) + + if f[1] == []: + # If there are no files listed, the user must be + # trying to create an empty directory, so just ignore + # it. + # Should we scream in this case? + continue + + # Check files one by one. + # Use revers order for safe removal + for j in range(len(f[1])-1, -1, -1): + data=f[1][j] + data = convert_path(data) + if not os.path.isfile(data): + # Again skip dirs. + continue + dst = os.path.join(dir, os.path.basename(data)) + if not os.path.exists(dst): + continue + del f[1][j] + if len(f[1]) == 0: + del self.data_files[i] + + return install_data.run(self) setup(name = 'usher', version = '0.1', package_dir = { 'usher':'python' }, packages = ['usher', 'usher.ushercli', 'usher.usherlnm', 'usher.utils', 'usher.usherctrl'], + cmdclass = {'install_data':smart_install_data}, data_files = [ ('/etc/init.d', ['initscripts/usherctrl', 'initscripts/usherlnm']), - ('/etc/usher', ['configs/usherctrl.config', 'configs/usherlnm.config', 'configs/ushercli.config']), + ('/etc/usher', ['configs/usherctrl.config', 'configs/usherlnm.config', 'configs/ushercli.config'], "preserve"), ("/usr/lib/python%s/site-packages/usher/usherctrl" % pyver, ['python/usherctrl/app.tac']), ("/usr/lib/python%s/site-packages/usher/usherlnm" % pyver, ['python/usherlnm/app.tac']) ] ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617496&group_id=5470 From noreply at sourceforge.net Sun Dec 17 21:33:11 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 12:33:11 -0800 Subject: [Patches] [ python-Patches-1617496 ] The ability to keep configuration files intact Message-ID: Patches item #1617496, was opened at 2006-12-17 21:02 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617496&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils and setup.py Group: None >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Roman Kurakin (sagamore) Assigned to: Nobody/Anonymous (nobody) Summary: The ability to keep configuration files intact Initial Comment: While working on one project I've met a problem that I can't to specify to not tuch my configuration files for python package that uses setup.py technique. To solve this problem I've implemented the following solution that works for me. If this idea worths it, I could made a pacth relative needed version of python. +class smart_install_data(install_data): + def run(self): + # Use revers order for safe removal + for i in range(len(self.data_files)-1, -1, -1): + f = self.data_files[i] + if type(f) is StringType: + continue + if len(f) <= 2: + continue + # Ok, we have additional value, do some magick according it. + if f[2] != "preserve": + # Nope, we do not know it. Just ignore for now. + # Should we scream about it? + continue + # Check a tuple with path to install to and a list of files + dir = convert_path(f[0]) + if not os.path.isabs(dir): + dir = os.path.join(self.install_dir, dir) + elif self.root: + dir = change_root(self.root, dir) + + if f[1] == []: + # If there are no files listed, the user must be + # trying to create an empty directory, so just ignore + # it. + # Should we scream in this case? + continue + + # Check files one by one. + # Use revers order for safe removal + for j in range(len(f[1])-1, -1, -1): + data=f[1][j] + data = convert_path(data) + if not os.path.isfile(data): + # Again skip dirs. + continue + dst = os.path.join(dir, os.path.basename(data)) + if not os.path.exists(dst): + continue + del f[1][j] + if len(f[1]) == 0: + del self.data_files[i] + + return install_data.run(self) setup(name = 'usher', version = '0.1', package_dir = { 'usher':'python' }, packages = ['usher', 'usher.ushercli', 'usher.usherlnm', 'usher.utils', 'usher.usherctrl'], + cmdclass = {'install_data':smart_install_data}, data_files = [ ('/etc/init.d', ['initscripts/usherctrl', 'initscripts/usherlnm']), - ('/etc/usher', ['configs/usherctrl.config', 'configs/usherlnm.config', 'configs/ushercli.config']), + ('/etc/usher', ['configs/usherctrl.config', 'configs/usherlnm.config', 'configs/ushercli.config'], "preserve"), ("/usr/lib/python%s/site-packages/usher/usherctrl" % pyver, ['python/usherctrl/app.tac']), ("/usr/lib/python%s/site-packages/usher/usherlnm" % pyver, ['python/usherlnm/app.tac']) ] ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-17 21:33 Message: Logged In: YES user_id=21627 Originator: NO Please suggest this feature to the distutils-sig first (http://www.python.org/community/sigs/current/distutils-sig/) After collecting feedback, please resubmit it as a new patch, and use a file attachment then, rather than including it in the message itself (it is incomprehensible as it stands). Also please include documentation changes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617496&group_id=5470 From noreply at sourceforge.net Mon Dec 18 04:09:49 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 19:09:49 -0800 Subject: [Patches] [ python-Patches-1617678 ] extended slicing for bytes objects Message-ID: Patches item #1617678, was opened at 2006-12-18 04:09 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617678&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Guido van Rossum (gvanrossum) Summary: extended slicing for bytes objects Initial Comment: >From the p3yk-noslice branch: extended slicing support for bytes objects. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617678&group_id=5470 From noreply at sourceforge.net Mon Dec 18 04:14:15 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 19:14:15 -0800 Subject: [Patches] [ python-Patches-1617680 ] slice-object support for sre_parse.SubPattern Message-ID: Patches item #1617680, was opened at 2006-12-18 04:14 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617680&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Fredrik Lundh (effbot) Summary: slice-object support for sre_parse.SubPattern Initial Comment: Support for sre_parse.SubPattern indexing by slice object, by simply delegating to the contained list. (backported from the p3yk-noslice branch.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617680&group_id=5470 From noreply at sourceforge.net Mon Dec 18 04:23:04 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 19:23:04 -0800 Subject: [Patches] [ python-Patches-1617682 ] specialcase simple sliceobj in tuple/str/unicode Message-ID: Patches item #1617682, was opened at 2006-12-18 04:23 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617682&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Nobody/Anonymous (nobody) Summary: specialcase simple sliceobj in tuple/str/unicode Initial Comment: Specialcase the step=1 (and step-None) case of slice-object handling in tuple, string and unicode objects -- like it is already specialcased for simple slices. (backported from the p3yk-noslice branch.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617682&group_id=5470 From noreply at sourceforge.net Mon Dec 18 04:35:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 19:35:42 -0800 Subject: [Patches] [ python-Patches-1617687 ] specialcase simple sliceobj in list (and bugfixes) Message-ID: Patches item #1617687, was opened at 2006-12-18 04:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617687&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Nobody/Anonymous (nobody) Summary: specialcase simple sliceobj in list (and bugfixes) Initial Comment: - Specialcase the step=1/None case of slicing using sliceobjects. - Fix bug where slice insertion using a sliceobject was inserting in a different place than slice inserting using a simple slice - Fix two off-by-one bugs that were cancelling each other out: the non-step-1 slice deletion code was memmoving too much for each item to delete, and not enough for the tail end, net result being a few too many bytes moved for each item. - Rewrite tail end move when deleting complex slice, use single memmove() instead of repeated PyList_SET_ITEM() - Expand comments describing the code somewhat. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617687&group_id=5470 From noreply at sourceforge.net Mon Dec 18 04:42:16 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 19:42:16 -0800 Subject: [Patches] [ python-Patches-1617691 ] Extended slicing for UserString Message-ID: Patches item #1617691, was opened at 2006-12-18 04:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617691&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Nobody/Anonymous (nobody) Summary: Extended slicing for UserString Initial Comment: - Add extended slicing support to UserString.UserString. - Add support for step-1 slice assignment using slice objects to UserString.MutableString. - Add tests for extended slicing support to all string_tests (now that all string types support it) - Add tests for the limited extended slice assignment support in MutableString. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617691&group_id=5470 From noreply at sourceforge.net Mon Dec 18 05:24:05 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 20:24:05 -0800 Subject: [Patches] [ python-Patches-1617698 ] Extended slicing for array objects Message-ID: Patches item #1617698, was opened at 2006-12-18 05:24 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617698&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Nobody/Anonymous (nobody) Summary: Extended slicing for array objects Initial Comment: Rework extended slicing for array objects, fixing a few bugs (like 'x[5:2] = y' working but 'x[5:2:1] = y' failing.) Optimize the common case of step-1 slicing. Also make extended slicing independent from simple slicing for easier removal of simple slicing in Py3K. Add tests for all extended slicing behaviour. (Backported from p3yk-noslice branch.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617698&group_id=5470 From noreply at sourceforge.net Mon Dec 18 05:28:01 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 20:28:01 -0800 Subject: [Patches] [ python-Patches-1617699 ] slice-object support for ctypes Pointer/Array Message-ID: Patches item #1617699, was opened at 2006-12-18 05:28 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617699&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Thomas Heller (theller) Summary: slice-object support for ctypes Pointer/Array Initial Comment: Support for slicing ctypes' Pointer and Array types with slice objects, although only for step=1 case. (Backported from p3yk-noslice branch.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617699&group_id=5470 From noreply at sourceforge.net Mon Dec 18 05:31:15 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 20:31:15 -0800 Subject: [Patches] [ python-Patches-1617700 ] slice-object support for mmap Message-ID: Patches item #1617700, was opened at 2006-12-18 05:31 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617700&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Nobody/Anonymous (nobody) Summary: slice-object support for mmap Initial Comment: Slice-object support for mmap.mmap objects, including non-step-1 support. (Backported from p3yk-noslice branch.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617700&group_id=5470 From noreply at sourceforge.net Mon Dec 18 05:32:50 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 20:32:50 -0800 Subject: [Patches] [ python-Patches-1617701 ] extended slicing for structseq Message-ID: Patches item #1617701, was opened at 2006-12-18 05:32 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Nobody/Anonymous (nobody) Summary: extended slicing for structseq Initial Comment: Extended slicing support for structseq objects (as returned by time.time() and os.stat()). Of little practical use, but necessary in p3yk. (Backported from p3yk-noslice branch.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617701&group_id=5470 From noreply at sourceforge.net Mon Dec 18 05:45:45 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 17 Dec 2006 20:45:45 -0800 Subject: [Patches] [ python-Patches-1617702 ] extended slicing for buffer objects Message-ID: Patches item #1617702, was opened at 2006-12-18 05:45 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617702&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Nobody/Anonymous (nobody) Summary: extended slicing for buffer objects Initial Comment: extended slicing support for buffer objects. Including slice assignment, but I don't know of a way to test assignment. (Backported from p3yk-noslice branch.) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617702&group_id=5470 From noreply at sourceforge.net Mon Dec 18 20:26:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 18 Dec 2006 11:26:32 -0800 Subject: [Patches] [ python-Patches-1615868 ] BZ2File.seek() fails for large files Message-ID: Patches item #1615868, was opened at 2006-12-14 11:25 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615868&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) >Assigned to: A.M. Kuchling (akuchling) Summary: BZ2File.seek() fails for large files Initial Comment: The bytesread variable in BZ2File.seek() is defined as int instead of size_t making it impossible to seek beyond 2GB distances. Affected are all Python versions. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-18 14:26 Message: Logged In: YES user_id=11375 Originator: NO Thanks for the patch! Committed to the trunk in rev. 53053 and to 25-maint in rev. 53054. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-14 11:45 Message: Logged In: YES user_id=642936 Originator: YES Nah, it must be Py_off_t. File Added: bz2-seek.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615868&group_id=5470 From noreply at sourceforge.net Mon Dec 18 20:29:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 18 Dec 2006 11:29:25 -0800 Subject: [Patches] [ python-Patches-1617678 ] extended slicing for bytes objects Message-ID: Patches item #1617678, was opened at 2006-12-17 22:09 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617678&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Guido van Rossum (gvanrossum) Summary: extended slicing for bytes objects Initial Comment: >From the p3yk-noslice branch: extended slicing support for bytes objects. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-18 14:29 Message: Logged In: YES user_id=6380 Originator: NO There are a few lines with trailing whitespace but otherwise I trust you. Go ahead! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617678&group_id=5470 From noreply at sourceforge.net Mon Dec 18 20:33:56 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 18 Dec 2006 11:33:56 -0800 Subject: [Patches] [ python-Patches-1613500 ] Write mode option for fileinput module. Message-ID: Patches item #1613500, was opened at 2006-12-11 21:16 Message generated for change (Comment added) made by antroy You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613500&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Anthony Roy (antroy) Assigned to: Nobody/Anonymous (nobody) Summary: Write mode option for fileinput module. Initial Comment: The current implementation of the fileinput module allows an optional 'mode' keyword argumant to the FileInput initializer and the input() convenience function. The attached patch provides a similar write_mode keyword argument with which to specify the write mode when the 'inplace' option is used. Included in the patch file are: 1) Changes to the fileinput module to accomodate the optional write mode option 2) Additions to the fileinput tex documentation. 3) A refactored test module, where the tests have been reorganised into TestCase subclasses, and new tests added for the new functionality. ---------------------------------------------------------------------- >Comment By: Anthony Roy (antroy) Date: 2006-12-18 19:33 Message: Logged In: YES user_id=1666022 Originator: YES The restriction isn't entirely arbitrary as far as I can see. If you allow the combination of inplace and openhook, then you will also need a writehook option, or the ability to specify a pair of hook function for the openhook. For example, there is a function provided for opening a compressed file for reading. In order to do an inplace edit, you would need to provide a write hook to compress the data on the way back in. I think that the implementation of such a change would be open for discussion, and is beyond the scope of a minor change as suggested in this patch. (Perhaps the topic for a future PEP). ---------------------------------------------------------------------- Comment By: Martin Kammerhofer (mkam) Date: 2006-12-15 13:10 Message: Logged In: YES user_id=1656067 Originator: NO The restriction that you cannot use inplace and openhook together seems somewhat arbitrary to me. Have you considered lifting it? From looking at the source it seems not too hard to do. There are uses for e.g. inplace editing compressed files. This would fit nicely with your patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613500&group_id=5470 From noreply at sourceforge.net Mon Dec 18 21:57:56 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 18 Dec 2006 12:57:56 -0800 Subject: [Patches] [ python-Patches-1617680 ] slice-object support for sre_parse.SubPattern Message-ID: Patches item #1617680, was opened at 2006-12-18 04:14 Message generated for change (Comment added) made by effbot You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617680&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) >Assigned to: Thomas Wouters (twouters) Summary: slice-object support for sre_parse.SubPattern Initial Comment: Support for sre_parse.SubPattern indexing by slice object, by simply delegating to the contained list. (backported from the p3yk-noslice branch.) ---------------------------------------------------------------------- >Comment By: Fredrik Lundh (effbot) Date: 2006-12-18 21:57 Message: Logged In: YES user_id=38376 Originator: NO Looks like a no-brainer. Go ahead and check it in. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617680&group_id=5470 From noreply at sourceforge.net Tue Dec 19 02:01:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 18 Dec 2006 17:01:42 -0800 Subject: [Patches] [ python-Patches-1618485 ] add None values in SimpleXMLRPCServer Message-ID: Patches item #1618485, was opened at 2006-12-19 01:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1618485&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Maximiliano Curia (maxy) Assigned to: Nobody/Anonymous (nobody) Summary: add None values in SimpleXMLRPCServer Initial Comment: Using SimpleXMLRPCServer I've been having some problems when trying to use "None" values. Apparently it uses a "dumps" function from xmlrpclib to generate the response. This function does support "None" values, but needs an extra parameter to allow them. The patch attached adds a new parameter (allow_none) to the constructor of SimpleXMLRPCServer, that when set is passed to the xmlrpclib.dumps call. This fixes the problem in a consistent way with the usage of xmlrpclib. It might be a better solution to support "None" values by default in SimpleXMLRPCServer and xmlrpclib. The patch is for python 2.4, but should probably be ported to 2.5 and 2.6. Thanks. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1618485&group_id=5470 From noreply at sourceforge.net Tue Dec 19 09:32:27 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 00:32:27 -0800 Subject: [Patches] [ python-Patches-1617678 ] extended slicing for bytes objects Message-ID: Patches item #1617678, was opened at 2006-12-18 04:09 Message generated for change (Comment added) made by twouters You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617678&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Guido van Rossum (gvanrossum) Summary: extended slicing for bytes objects Initial Comment: >From the p3yk-noslice branch: extended slicing support for bytes objects. ---------------------------------------------------------------------- >Comment By: Thomas Wouters (twouters) Date: 2006-12-19 09:32 Message: Logged In: YES user_id=34209 Originator: YES checked in. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-18 20:29 Message: Logged In: YES user_id=6380 Originator: NO There are a few lines with trailing whitespace but otherwise I trust you. Go ahead! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617678&group_id=5470 From noreply at sourceforge.net Tue Dec 19 09:33:18 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 00:33:18 -0800 Subject: [Patches] [ python-Patches-1617680 ] slice-object support for sre_parse.SubPattern Message-ID: Patches item #1617680, was opened at 2006-12-18 04:14 Message generated for change (Comment added) made by twouters You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617680&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 >Status: Closed >Resolution: Fixed Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Thomas Wouters (twouters) Summary: slice-object support for sre_parse.SubPattern Initial Comment: Support for sre_parse.SubPattern indexing by slice object, by simply delegating to the contained list. (backported from the p3yk-noslice branch.) ---------------------------------------------------------------------- >Comment By: Thomas Wouters (twouters) Date: 2006-12-19 09:33 Message: Logged In: YES user_id=34209 Originator: YES Checked in. ---------------------------------------------------------------------- Comment By: Fredrik Lundh (effbot) Date: 2006-12-18 21:57 Message: Logged In: YES user_id=38376 Originator: NO Looks like a no-brainer. Go ahead and check it in. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617680&group_id=5470 From noreply at sourceforge.net Tue Dec 19 11:48:08 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 02:48:08 -0800 Subject: [Patches] [ python-Patches-1615158 ] POSIX capabilities support Message-ID: Patches item #1615158, was opened at 2006-12-13 18:10 Message generated for change (Comment added) made by gj0aqzda You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Matt Kern (gj0aqzda) Assigned to: Nobody/Anonymous (nobody) Summary: POSIX capabilities support Initial Comment: Attached is a patch which adds POSIX capabilities support. The following API functions are supported: * cap_clear * cap_copy_ext * cap_dup * cap_from_text * cap_get_flag * cap_get_proc * cap_init * cap_set_flag * cap_set_proc * cap_size * cap_to_text The following API function is supported, but is broken with certain versions of libcap (I am running debian testing's libcap1, version 1.10-14, which has an issue; I have reported this upstream): * cap_copy_int The following API functions are in there as stubs, but currently are not compiled. I need access to a machine to test these. I will probably add autoconf tests for availability of these functions in due course: * cap_get_fd * cap_get_file * cap_set_fd * cap_set_file The patch includes diffs to configure. My autoconf is however at a different revision to that used on the python trunk. You may want to re-autoconf configure.in. I've added a few API tests to test_posix.py. ---------------------------------------------------------------------- >Comment By: Matt Kern (gj0aqzda) Date: 2006-12-19 10:48 Message: Logged In: YES user_id=1667774 Originator: YES I've attached a documentation patch, which should be applied in addition to the base patch. File Added: patch-svn-doc.diff ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-16 13:25 Message: Logged In: YES user_id=849994 Originator: NO (If you don't want to write LaTeX, it's enough to write the docs in plaintext, there are a few volunteers who will convert it appropriately.) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 12:28 Message: Logged In: YES user_id=21627 Originator: NO Can you please provide documentation changes as well? ---------------------------------------------------------------------- Comment By: Matt Kern (gj0aqzda) Date: 2006-12-13 18:12 Message: Logged In: YES user_id=1667774 Originator: YES I should further add that I have implemented the following API calls as methods of the new CapabilityState object in addition to the standard functions: * cap_clear * cap_copy_ext * cap_dup * cap_get_flag * cap_set_flag * cap_set_proc * cap_size * cap_to_text ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615158&group_id=5470 From noreply at sourceforge.net Tue Dec 19 16:14:15 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 07:14:15 -0800 Subject: [Patches] [ python-Patches-1617413 ] urllib HTTPS Basic authentication fix Message-ID: Patches item #1617413, was opened at 2006-12-17 11:05 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617413&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Dug Song (dugsong) >Assigned to: A.M. Kuchling (akuchling) Summary: urllib HTTPS Basic authentication fix Initial Comment: urllib's support for HTTP Basic authentication via HTTPS is broken, due to an invalid call in URLopener.open_https() to HTTPConnection.putheader() (apparently miscopied from URLopener.open_http(), which gets it right). the code incorrectly calls putheader with a fully formatted header instead of passing a name and value separately, resulting in a badly formatted header ending with a trailing colon. the fix is trivial. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-19 10:14 Message: Logged In: YES user_id=11375 Originator: NO Thanks for the fix! Applied to trunk in rev. 53068 and to 25-maint in rev. 53069. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617413&group_id=5470 From noreply at sourceforge.net Tue Dec 19 16:19:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 07:19:32 -0800 Subject: [Patches] [ python-Patches-1600491 ] 1572210 doc patch Message-ID: Patches item #1600491, was opened at 2006-11-21 10:38 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1600491&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: Library (Lib) Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Jim Jewett (jimjjewett) >Assigned to: A.M. Kuchling (akuchling) Summary: 1572210 doc patch Initial Comment: help on a keyword doesn't work, unless the help files are built. On at least windows, they are not built by default. This adds instructions on how to build them. C:\Python25\Doc>hh -decompile . Python25.chm ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-19 10:19 Message: Logged In: YES user_id=11375 Originator: NO Thanks! Applied to trunk in rev. 53071 and 25-maint in 53072. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1600491&group_id=5470 From noreply at sourceforge.net Tue Dec 19 16:44:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 07:44:25 -0800 Subject: [Patches] [ python-Patches-1587139 ] cookielib: lock acquire/release try..finally protected Message-ID: Patches item #1587139, was opened at 2006-10-30 06:39 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1587139&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: kxroberto (kxroberto) >Assigned to: A.M. Kuchling (akuchling) Summary: cookielib: lock acquire/release try..finally protected Initial Comment: Almost all code between lock.acquire and .relase in cookielib was unprotected by a try..finally bracket. I suspect some deadlocks here to have to do with that. This patch against latest version (2.5) in SVN corrects it. Another minor change request: at the end of cookielib.py these 2 special CookieJar-modules are just imported without being referenced/used in cookielib: from _LWPCookieJar import LWPCookieJar, lwp_cookie_str from _MozillaCookieJar import MozillaCookieJar Maybe that should be removed from cookielib in order to further reduce the slow import and leave the (rarely used) import to the user (as with BSDJar etc. too ...) robert ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-19 10:44 Message: Logged In: YES user_id=11375 Originator: NO Thanks for your patch! Applied to the trunk in rev. 53073. We probably can't remove the imports at the end because they're for preserving backward compatibility with earlier versions of cookielib that contained those classes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1587139&group_id=5470 From noreply at sourceforge.net Tue Dec 19 18:50:57 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 09:50:57 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 08:08 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 12:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Tue Dec 19 19:55:20 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 10:55:20 -0800 Subject: [Patches] [ python-Patches-1601678 ] sys.intern() Message-ID: Patches item #1601678, was opened at 2006-11-23 11:10 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1601678&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) >Summary: sys.intern() Initial Comment: This patch moves id() and intern() into the sys module, as listed in PEP 3100. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 18:55 Message: Logged In: YES user_id=849994 Originator: YES Updated patch only moves intern(), as per python-3000 discussion. File Added: sys-intern.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-11-23 16:43 Message: Logged In: YES user_id=6380 Originator: NO Aargh. I need to think more about whether I even want this. Doesn't PEP 3100 have a ? near that item? Everything with a ? needs to be reconsidered before implemented. Add a ? if it's not already there for this issue. Sorry. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1601678&group_id=5470 From noreply at sourceforge.net Tue Dec 19 20:23:20 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 11:23:20 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 14:08 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 20:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 18:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Tue Dec 19 20:26:55 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 11:26:55 -0800 Subject: [Patches] [ python-Patches-1601678 ] sys.intern() Message-ID: Patches item #1601678, was opened at 2006-11-23 06:10 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1601678&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.intern() Initial Comment: This patch moves id() and intern() into the sys module, as listed in PEP 3100. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 14:26 Message: Logged In: YES user_id=6380 Originator: NO Looks good, check it in (and update PEP 3100 please). Were there no other places where intern() is actually *used* in the stdlib? Amazing! Can I convince you to also contribute a fixer for the sandbox/2to3 refactoring tool? ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 13:55 Message: Logged In: YES user_id=849994 Originator: YES Updated patch only moves intern(), as per python-3000 discussion. File Added: sys-intern.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-11-23 11:43 Message: Logged In: YES user_id=6380 Originator: NO Aargh. I need to think more about whether I even want this. Doesn't PEP 3100 have a ? near that item? Everything with a ? needs to be reconsidered before implemented. Add a ? if it's not already there for this issue. Sorry. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1601678&group_id=5470 From noreply at sourceforge.net Tue Dec 19 21:54:06 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 12:54:06 -0800 Subject: [Patches] [ python-Patches-1619049 ] sys.intern() 2to3 fixer Message-ID: Patches item #1619049, was opened at 2006-12-19 20:54 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.intern() 2to3 fixer Initial Comment: This is a fixer for the 2to3 refactoring utility, replacing occurences of intern() by sys.intern(). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 From noreply at sourceforge.net Tue Dec 19 21:54:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 12:54:32 -0800 Subject: [Patches] [ python-Patches-1601678 ] sys.intern() Message-ID: Patches item #1601678, was opened at 2006-11-23 11:10 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1601678&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 3000 >Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.intern() Initial Comment: This patch moves id() and intern() into the sys module, as listed in PEP 3100. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 20:54 Message: Logged In: YES user_id=849994 Originator: YES Committed in rev. 53080. Updated the PEP in rev. 53082. Submitted new patch for the 2to3 refactoring tool in #1619049. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 19:26 Message: Logged In: YES user_id=6380 Originator: NO Looks good, check it in (and update PEP 3100 please). Were there no other places where intern() is actually *used* in the stdlib? Amazing! Can I convince you to also contribute a fixer for the sandbox/2to3 refactoring tool? ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 18:55 Message: Logged In: YES user_id=849994 Originator: YES Updated patch only moves intern(), as per python-3000 discussion. File Added: sys-intern.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-11-23 16:43 Message: Logged In: YES user_id=6380 Originator: NO Aargh. I need to think more about whether I even want this. Doesn't PEP 3100 have a ? near that item? Everything with a ? needs to be reconsidered before implemented. Add a ? if it's not already there for this issue. Sorry. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1601678&group_id=5470 From noreply at sourceforge.net Tue Dec 19 21:55:48 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 12:55:48 -0800 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 13:51 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 7 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 20:55 Message: Logged In: YES user_id=849994 Originator: NO What's the status of this patch? Does it still have to be applied somewhere? ---------------------------------------------------------------------- Comment By: Thomas Waldmann (thomaswaldmann) Date: 2006-09-30 19:28 Message: Logged In: YES user_id=100649 Could someone please backport this to 2.4 branch? Currently, 2.4.4c0 is broken when you try to extract a file that happens to have a "/" at index 99 in its name (tarinfo.name ends with "/" in this case, but this is NOT the end of the complete (LONG) filename). It gets misinterpreted as directory in this case. MoinMoin build process is broken on 2.4.4c0 due to this bug. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-10 00:23 Message: Logged In: YES user_id=33168 Ah, I see. I checked in the patch to remove the debug message since it's dead code. We are in feature freeze for 2.5, so I've updated the patch, attached it, set the group to 2.6. We can add the patch after 2.5 is out in a month or so. Committed revision 50503. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-09 08:01 Message: Logged In: YES user_id=642936 I attached a tar with two diffs: only-tarfile.diff only removes the debug message. complete-with-headererror.diff removes the debug message and adds the HeaderError exception to TarInfo.frombuf() mentioned below. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-04 22:04 Message: Logged In: YES user_id=33168 Lars, I don't see the debug (_dbg) call. Which version are you using? I didn't see r46040 in Lib/tafile.py for either 2.5 or 2.4. Could you point me to file/line/version where there's a problem (or just make a patch)? Thanks. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-03 08:59 Message: Logged In: YES user_id=642936 I would like to emphasize on point (2) of my previous post, which should be resolved before 2.5 comes out. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-18 10:00 Message: Logged In: YES user_id=642936 On (1): agreed. On (2): There is still a debug message emitted for a bad checksum: In TarInfo.frombuf() at the bottom a ValueError is raised if they don't match and is passed on to TarFile.next() where it is put out as a debug message using the _dbg() method in the except clause. The debug message where it is now (r46040) is senseless because the try-block will be left when TarInfo.frombuf() fails . On (3): You're right, I attached a patch that adds another Exception HeaderError which is raised in TarInfo.frombuf() instead of ValueError in case of a bad header. I hope that is acceptable. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-18 06:11 Message: Logged In: YES user_id=849994 Jim: I agree with your comments and have committed an improved version. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-05-17 22:25 Message: Logged In: YES user_id=764593 (1) Why change the exception style? When raising an instance, the style guide (PEP-8, http:// www.python.org/dev/peps/pep-0008/) prefers to construct that instance; the older form is left over from String exceptions and will be removed in Python 3. I could understand leaving them as they were, but if you're going to change them to make them consistent, why not use the current format? (2) Why get rid of the debug messages (such as the checksum check) entirely? Guarding them in if self.debug, I would understand. (3) I wouldn't count on str(e) (where e is any ValueError instance) being as meaningful as the (current version's) ReadError("empty, unreadable or compressed file") ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 16:26 Message: Logged In: YES user_id=849994 Thanks for the patch, applied as rev. 45954. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 13:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:04:19 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:04:19 -0800 Subject: [Patches] [ python-Patches-1613352 ] encoding directive -- more examples Message-ID: Patches item #1613352, was opened at 2006-12-11 17:22 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613352&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Jim Jewett (jimjjewett) Assigned to: Nobody/Anonymous (nobody) Summary: encoding directive -- more examples Initial Comment: As pointed out in http://groups.google.com/group/comp.lang.python/browse_frm/thread/c62220ff0f4cb30a many people read PEP 263 as having stricter requirements -- matching the format shown in the "defining the encoding" example. Make it clear that the regex is the only requirement, with additional examples and a slight wording change. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 21:04 Message: Logged In: YES user_id=849994 Originator: NO Applied a similar patch in rev. 53084. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1613352&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:06:29 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:06:29 -0800 Subject: [Patches] [ python-Patches-1619049 ] sys.intern() 2to3 fixer Message-ID: Patches item #1619049, was opened at 2006-12-19 15:54 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.intern() 2to3 fixer Initial Comment: This is a fixer for the 2to3 refactoring utility, replacing occurences of intern() by sys.intern(). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 16:06 Message: Logged In: YES user_id=6380 Originator: NO Pretty good. I think you used fix_apply.py as an example? That unfortunately doesn't handle the following case: apply(f, x, y).something The has_key() fixes *does* support that (using the "after=*any" pattern. Can you copy the relevant parts from that fixer? Also, what do you think of how easy it is to write a refactoring using this tool? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:07:00 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:07:00 -0800 Subject: [Patches] [ python-Patches-1591665 ] adding __dir__ Message-ID: Patches item #1591665, was opened at 2006-11-06 21:52 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1591665&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None >Priority: 6 Private: No Submitted By: ganges master (gangesmaster) Assigned to: Neal Norwitz (nnorwitz) Summary: adding __dir__ Initial Comment: in accordance with http://mail.python.org/pipermail/python-dev/2006-November/069865.html i've written a patch that allows objects to define their own introspection mechanisms, by providing __dir__. with this patch: * dir() returns the locals. this is done in builtin_dir() * dir(obj) returns the attributes of obj, by invoking PyObject_Dir() * if obj->ob_type has "__dir__", it is used. note that it must return a list! * otherwise, use default the mechanism of collecting attributes * for module objects, return __dict__.keys() * for type objects, return __dict__.keys() + dir(obj.__base__) * for all other objects, return __dict__.keys() + __members__ + __methods__ + dir(obj.__class__) * builtin_dir takes care of sorting the list ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2006-11-23 12:11 Message: Logged In: YES user_id=4771 Originator: NO Line 20 in demo.py: assert "__getitem__" in dir(x) looks strange to me... Foo doesn't inherit from any sequence or mapping type. ---------------------------------------------------------------------- Comment By: ganges master (gangesmaster) Date: 2006-11-11 21:31 Message: Logged In: YES user_id=1406776 > PyObject_CallFunctionObjArgs(dirfunc, obj, NULL) done > Couldn't __dir__ also be allowed to return a tuple? no, because tuples are not sortable, and i don't want to over complicate the c-side code of PyObject_Dir. having __dir__ returning only a list is equivalent to __repr__ returning only strings. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-11-11 19:58 Message: Logged In: YES user_id=849994 * Instead of doing PyObject_CallFunction(dirfunc, "O", obj) you should do PyObject_CallFunctionObjArgs(dirfunc, obj, NULL). * Couldn't __dir__ also be allowed to return a tuple? ---------------------------------------------------------------------- Comment By: ganges master (gangesmaster) Date: 2006-11-08 11:22 Message: Logged In: YES user_id=1406776 i like to init all my locals ("just in case"), but if the rest of the code does not adhere my style, i'll change that. anyway, i made the changes to the code, updated the docs, and added full tests (the original dir() wasn't test so thoroughly) -tomer ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-11-08 05:53 Message: Logged In: YES user_id=33168 tomer, do you know about configuring with --pydebug? That helps track down refleaks when running regrtest -R ::. object.c: _dir_locals: result is not necessary and locals doesn't need to be initialized as it's set on the next line. You could just declare and set it all on one line. _specialized_dir_type should be static. No need to init dict. Either don't init result or remove else result = NULL. I'd prefer removing the else and leaving the init. _specialized_dir_module should be static. No need to init dict. Can you get the name of the module and use that in the error msg: PyModule_GetName()? That would hopefully provide a nicer error msg. _generic_dir: No need to init dict. + /* XXX api change: how about falling to obj->ob_type + XXX if no __class__ exists? */ Do you mean falling *back*? Also, we've been using XXX(username): as the format for such comments. So this would be better as: /* XXX(tomer): Perhaps fall back to obj->ob_type if no __class__ exists? */ _dir_object: No need to init dirfunc. PyObject_Dir: No need to init result. Are there tests for all conditions? At least: * dir() * dir(obj) * dir(obj_with_no_dict) * dir(obj_with_no__class__) * dir(obj_with__methods__) * dir(obj_with__members__) * dir(module) * dir(module_with_no__dict__) * dir(module_with_invalid__dict__) There also need to be updates to Doc/lib/libfuncs.tex. If you can't deal with the markup, just do the best you can in text and someone else will fixup the markup. Thanks for attaching the patch as a single file, it's easier to deal with. ---------------------------------------------------------------------- Comment By: ganges master (gangesmaster) Date: 2006-11-07 15:37 Message: Logged In: YES user_id=1406776 okay: * builtin_dir directly calls PyObject_Dir * PyObject_Dir handles NULL argument and sorting * it is now completely compatible with the 2.5 API * fixed several refcount bugs (i wish we had a tracing gc :) ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2006-11-06 22:52 Message: Logged In: YES user_id=1038590 The retrieval of locals on a NULL argument and the sorting step need to move back inside PyObject_Dir to avoid changing the C API. If the standard library's current C API tests didn't break on this version of the patch, then the final version of the patch should include enhanced tests for PyObject_Dir that pass both before and after the patch is applied to PyObject_Dir. Other than that, I didn't see any major problems on reading the code (i.e. refcounts and error handling looked pretty reasonable). I haven't actually run it though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1591665&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:07:01 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:07:01 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 08:08 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 16:07 Message: Logged In: YES user_id=80475 Originator: NO I'm -1 on making ANY guarantees about which methods underlie others -- that would constitute new and everlasting guarantees about how mappings are implemented. Subclasses should explicity override/extend the methods withed changed behavior. If that proves non-trivial, then it is likely there should be a has-a relationship instead of an is-a relationship. Also, it is likely that the subclass will have Liskov substitutability violations. Either way, there is probably a design flaw. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 14:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 12:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:12:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:12:43 -0800 Subject: [Patches] [ python-Patches-1591665 ] adding __dir__ Message-ID: Patches item #1591665, was opened at 2006-11-06 23:52 Message generated for change (Comment added) made by gangesmaster You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1591665&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 6 Private: No Submitted By: ganges master (gangesmaster) Assigned to: Neal Norwitz (nnorwitz) Summary: adding __dir__ Initial Comment: in accordance with http://mail.python.org/pipermail/python-dev/2006-November/069865.html i've written a patch that allows objects to define their own introspection mechanisms, by providing __dir__. with this patch: * dir() returns the locals. this is done in builtin_dir() * dir(obj) returns the attributes of obj, by invoking PyObject_Dir() * if obj->ob_type has "__dir__", it is used. note that it must return a list! * otherwise, use default the mechanism of collecting attributes * for module objects, return __dict__.keys() * for type objects, return __dict__.keys() + dir(obj.__base__) * for all other objects, return __dict__.keys() + __members__ + __methods__ + dir(obj.__class__) * builtin_dir takes care of sorting the list ---------------------------------------------------------------------- >Comment By: ganges master (gangesmaster) Date: 2006-12-19 23:12 Message: Logged In: YES user_id=1406776 Originator: YES i guess the demo isn't updated/relevant anymore. instead, concrete tests were added to lib/tests/test_builtin.py ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2006-11-23 14:11 Message: Logged In: YES user_id=4771 Originator: NO Line 20 in demo.py: assert "__getitem__" in dir(x) looks strange to me... Foo doesn't inherit from any sequence or mapping type. ---------------------------------------------------------------------- Comment By: ganges master (gangesmaster) Date: 2006-11-11 23:31 Message: Logged In: YES user_id=1406776 > PyObject_CallFunctionObjArgs(dirfunc, obj, NULL) done > Couldn't __dir__ also be allowed to return a tuple? no, because tuples are not sortable, and i don't want to over complicate the c-side code of PyObject_Dir. having __dir__ returning only a list is equivalent to __repr__ returning only strings. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-11-11 21:58 Message: Logged In: YES user_id=849994 * Instead of doing PyObject_CallFunction(dirfunc, "O", obj) you should do PyObject_CallFunctionObjArgs(dirfunc, obj, NULL). * Couldn't __dir__ also be allowed to return a tuple? ---------------------------------------------------------------------- Comment By: ganges master (gangesmaster) Date: 2006-11-08 13:22 Message: Logged In: YES user_id=1406776 i like to init all my locals ("just in case"), but if the rest of the code does not adhere my style, i'll change that. anyway, i made the changes to the code, updated the docs, and added full tests (the original dir() wasn't test so thoroughly) -tomer ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-11-08 07:53 Message: Logged In: YES user_id=33168 tomer, do you know about configuring with --pydebug? That helps track down refleaks when running regrtest -R ::. object.c: _dir_locals: result is not necessary and locals doesn't need to be initialized as it's set on the next line. You could just declare and set it all on one line. _specialized_dir_type should be static. No need to init dict. Either don't init result or remove else result = NULL. I'd prefer removing the else and leaving the init. _specialized_dir_module should be static. No need to init dict. Can you get the name of the module and use that in the error msg: PyModule_GetName()? That would hopefully provide a nicer error msg. _generic_dir: No need to init dict. + /* XXX api change: how about falling to obj->ob_type + XXX if no __class__ exists? */ Do you mean falling *back*? Also, we've been using XXX(username): as the format for such comments. So this would be better as: /* XXX(tomer): Perhaps fall back to obj->ob_type if no __class__ exists? */ _dir_object: No need to init dirfunc. PyObject_Dir: No need to init result. Are there tests for all conditions? At least: * dir() * dir(obj) * dir(obj_with_no_dict) * dir(obj_with_no__class__) * dir(obj_with__methods__) * dir(obj_with__members__) * dir(module) * dir(module_with_no__dict__) * dir(module_with_invalid__dict__) There also need to be updates to Doc/lib/libfuncs.tex. If you can't deal with the markup, just do the best you can in text and someone else will fixup the markup. Thanks for attaching the patch as a single file, it's easier to deal with. ---------------------------------------------------------------------- Comment By: ganges master (gangesmaster) Date: 2006-11-07 17:37 Message: Logged In: YES user_id=1406776 okay: * builtin_dir directly calls PyObject_Dir * PyObject_Dir handles NULL argument and sorting * it is now completely compatible with the 2.5 API * fixed several refcount bugs (i wish we had a tracing gc :) ---------------------------------------------------------------------- Comment By: Nick Coghlan (ncoghlan) Date: 2006-11-07 00:52 Message: Logged In: YES user_id=1038590 The retrieval of locals on a NULL argument and the sorting step need to move back inside PyObject_Dir to avoid changing the C API. If the standard library's current C API tests didn't break on this version of the patch, then the final version of the patch should include enhanced tests for PyObject_Dir that pass both before and after the patch is applied to PyObject_Dir. Other than that, I didn't see any major problems on reading the code (i.e. refcounts and error handling looked pretty reasonable). I haven't actually run it though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1591665&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:24:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:24:32 -0800 Subject: [Patches] [ python-Patches-1619049 ] sys.intern() 2to3 fixer Message-ID: Patches item #1619049, was opened at 2006-12-19 20:54 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.intern() 2to3 fixer Initial Comment: This is a fixer for the 2to3 refactoring utility, replacing occurences of intern() by sys.intern(). ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 21:24 Message: Logged In: YES user_id=849994 Originator: YES Yes, I looked at fix_apply and didn't think about additional trailers. Attaching updated patch. I think once you get the general idea, it's fairly easy to write refactorings. (I was only stuck for a while figuring out that only one argument doesn't create an "arglist" node at all). File Added: 2to3-intern-2.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 21:06 Message: Logged In: YES user_id=6380 Originator: NO Pretty good. I think you used fix_apply.py as an example? That unfortunately doesn't handle the following case: apply(f, x, y).something The has_key() fixes *does* support that (using the "after=*any" pattern. Can you copy the relevant parts from that fixer? Also, what do you think of how easy it is to write a refactoring using this tool? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:39:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:39:07 -0800 Subject: [Patches] [ python-Patches-1619049 ] sys.intern() 2to3 fixer Message-ID: Patches item #1619049, was opened at 2006-12-19 15:54 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.intern() 2to3 fixer Initial Comment: This is a fixer for the 2to3 refactoring utility, replacing occurences of intern() by sys.intern(). ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 16:39 Message: Logged In: YES user_id=6380 Originator: NO OK, check it in. We need to figure out one more thing eventually -- how to insert the needed "import sys" if it isn't there already. But let's put that off a bit. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 16:24 Message: Logged In: YES user_id=849994 Originator: YES Yes, I looked at fix_apply and didn't think about additional trailers. Attaching updated patch. I think once you get the general idea, it's fairly easy to write refactorings. (I was only stuck for a while figuring out that only one argument doesn't create an "arglist" node at all). File Added: 2to3-intern-2.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 16:06 Message: Logged In: YES user_id=6380 Originator: NO Pretty good. I think you used fix_apply.py as an example? That unfortunately doesn't handle the following case: apply(f, x, y).something The has_key() fixes *does* support that (using the "after=*any" pattern. Can you copy the relevant parts from that fixer? Also, what do you think of how easy it is to write a refactoring using this tool? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:44:18 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:44:18 -0800 Subject: [Patches] [ python-Patches-1619049 ] sys.intern() 2to3 fixer Message-ID: Patches item #1619049, was opened at 2006-12-19 20:54 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 3000 >Status: Closed Resolution: Accepted Priority: 5 Private: No Submitted By: Georg Brandl (gbrandl) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.intern() 2to3 fixer Initial Comment: This is a fixer for the 2to3 refactoring utility, replacing occurences of intern() by sys.intern(). ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 21:44 Message: Logged In: YES user_id=849994 Originator: YES Okay, applied as rev. 53088. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 21:39 Message: Logged In: YES user_id=6380 Originator: NO OK, check it in. We need to figure out one more thing eventually -- how to insert the needed "import sys" if it isn't there already. But let's put that off a bit. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 21:24 Message: Logged In: YES user_id=849994 Originator: YES Yes, I looked at fix_apply and didn't think about additional trailers. Attaching updated patch. I think once you get the general idea, it's fairly easy to write refactorings. (I was only stuck for a while figuring out that only one argument doesn't create an "arglist" node at all). File Added: 2to3-intern-2.diff ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 21:06 Message: Logged In: YES user_id=6380 Originator: NO Pretty good. I think you used fix_apply.py as an example? That unfortunately doesn't handle the following case: apply(f, x, y).something The has_key() fixes *does* support that (using the "after=*any" pattern. Can you copy the relevant parts from that fixer? Also, what do you think of how easy it is to write a refactoring using this tool? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619049&group_id=5470 From noreply at sourceforge.net Tue Dec 19 22:50:48 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 13:50:48 -0800 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 15:51 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 7 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-19 22:50 Message: Logged In: YES user_id=642936 Originator: YES Yes, please apply complete-with-headererror.diff to the trunk. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 21:55 Message: Logged In: YES user_id=849994 Originator: NO What's the status of this patch? Does it still have to be applied somewhere? ---------------------------------------------------------------------- Comment By: Thomas Waldmann (thomaswaldmann) Date: 2006-09-30 21:28 Message: Logged In: YES user_id=100649 Could someone please backport this to 2.4 branch? Currently, 2.4.4c0 is broken when you try to extract a file that happens to have a "/" at index 99 in its name (tarinfo.name ends with "/" in this case, but this is NOT the end of the complete (LONG) filename). It gets misinterpreted as directory in this case. MoinMoin build process is broken on 2.4.4c0 due to this bug. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-10 02:23 Message: Logged In: YES user_id=33168 Ah, I see. I checked in the patch to remove the debug message since it's dead code. We are in feature freeze for 2.5, so I've updated the patch, attached it, set the group to 2.6. We can add the patch after 2.5 is out in a month or so. Committed revision 50503. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-09 10:01 Message: Logged In: YES user_id=642936 I attached a tar with two diffs: only-tarfile.diff only removes the debug message. complete-with-headererror.diff removes the debug message and adds the HeaderError exception to TarInfo.frombuf() mentioned below. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-05 00:04 Message: Logged In: YES user_id=33168 Lars, I don't see the debug (_dbg) call. Which version are you using? I didn't see r46040 in Lib/tafile.py for either 2.5 or 2.4. Could you point me to file/line/version where there's a problem (or just make a patch)? Thanks. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-03 10:59 Message: Logged In: YES user_id=642936 I would like to emphasize on point (2) of my previous post, which should be resolved before 2.5 comes out. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-18 12:00 Message: Logged In: YES user_id=642936 On (1): agreed. On (2): There is still a debug message emitted for a bad checksum: In TarInfo.frombuf() at the bottom a ValueError is raised if they don't match and is passed on to TarFile.next() where it is put out as a debug message using the _dbg() method in the except clause. The debug message where it is now (r46040) is senseless because the try-block will be left when TarInfo.frombuf() fails . On (3): You're right, I attached a patch that adds another Exception HeaderError which is raised in TarInfo.frombuf() instead of ValueError in case of a bad header. I hope that is acceptable. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-18 08:11 Message: Logged In: YES user_id=849994 Jim: I agree with your comments and have committed an improved version. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-05-18 00:25 Message: Logged In: YES user_id=764593 (1) Why change the exception style? When raising an instance, the style guide (PEP-8, http:// www.python.org/dev/peps/pep-0008/) prefers to construct that instance; the older form is left over from String exceptions and will be removed in Python 3. I could understand leaving them as they were, but if you're going to change them to make them consistent, why not use the current format? (2) Why get rid of the debug messages (such as the checksum check) entirely? Guarding them in if self.debug, I would understand. (3) I wouldn't count on str(e) (where e is any ValueError instance) being as meaningful as the (current version's) ReadError("empty, unreadable or compressed file") ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 18:26 Message: Logged In: YES user_id=849994 Thanks for the patch, applied as rev. 45954. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 15:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Tue Dec 19 23:07:04 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 14:07:04 -0800 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 13:51 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 >Status: Closed >Resolution: Accepted Priority: 7 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 22:07 Message: Logged In: YES user_id=849994 Originator: NO Applied in rev. 53090. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-19 21:50 Message: Logged In: YES user_id=642936 Originator: YES Yes, please apply complete-with-headererror.diff to the trunk. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 20:55 Message: Logged In: YES user_id=849994 Originator: NO What's the status of this patch? Does it still have to be applied somewhere? ---------------------------------------------------------------------- Comment By: Thomas Waldmann (thomaswaldmann) Date: 2006-09-30 19:28 Message: Logged In: YES user_id=100649 Could someone please backport this to 2.4 branch? Currently, 2.4.4c0 is broken when you try to extract a file that happens to have a "/" at index 99 in its name (tarinfo.name ends with "/" in this case, but this is NOT the end of the complete (LONG) filename). It gets misinterpreted as directory in this case. MoinMoin build process is broken on 2.4.4c0 due to this bug. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-10 00:23 Message: Logged In: YES user_id=33168 Ah, I see. I checked in the patch to remove the debug message since it's dead code. We are in feature freeze for 2.5, so I've updated the patch, attached it, set the group to 2.6. We can add the patch after 2.5 is out in a month or so. Committed revision 50503. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-09 08:01 Message: Logged In: YES user_id=642936 I attached a tar with two diffs: only-tarfile.diff only removes the debug message. complete-with-headererror.diff removes the debug message and adds the HeaderError exception to TarInfo.frombuf() mentioned below. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-04 22:04 Message: Logged In: YES user_id=33168 Lars, I don't see the debug (_dbg) call. Which version are you using? I didn't see r46040 in Lib/tafile.py for either 2.5 or 2.4. Could you point me to file/line/version where there's a problem (or just make a patch)? Thanks. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-03 08:59 Message: Logged In: YES user_id=642936 I would like to emphasize on point (2) of my previous post, which should be resolved before 2.5 comes out. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-18 10:00 Message: Logged In: YES user_id=642936 On (1): agreed. On (2): There is still a debug message emitted for a bad checksum: In TarInfo.frombuf() at the bottom a ValueError is raised if they don't match and is passed on to TarFile.next() where it is put out as a debug message using the _dbg() method in the except clause. The debug message where it is now (r46040) is senseless because the try-block will be left when TarInfo.frombuf() fails . On (3): You're right, I attached a patch that adds another Exception HeaderError which is raised in TarInfo.frombuf() instead of ValueError in case of a bad header. I hope that is acceptable. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-18 06:11 Message: Logged In: YES user_id=849994 Jim: I agree with your comments and have committed an improved version. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-05-17 22:25 Message: Logged In: YES user_id=764593 (1) Why change the exception style? When raising an instance, the style guide (PEP-8, http:// www.python.org/dev/peps/pep-0008/) prefers to construct that instance; the older form is left over from String exceptions and will be removed in Python 3. I could understand leaving them as they were, but if you're going to change them to make them consistent, why not use the current format? (2) Why get rid of the debug messages (such as the checksum check) entirely? Guarding them in if self.debug, I would understand. (3) I wouldn't count on str(e) (where e is any ValueError instance) being as meaningful as the (current version's) ReadError("empty, unreadable or compressed file") ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 16:26 Message: Logged In: YES user_id=849994 Thanks for the patch, applied as rev. 45954. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 13:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Tue Dec 19 23:26:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 14:26:53 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 08:08 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 17:26 Message: Logged In: YES user_id=764593 Originator: NO As I understand it, the problem is that dict.update is assuming any dict subclass will use the same internal data representation. Restricting the fast path to exactly builtin dicts (not subclasses) fixes the bug, but makes the fallback more frequent. The existing fallback is to call keys(), then iterate over it, retrieving the value for each key. (keys is required for a "minimal mapping" as documented is UserDict, and a few other random places.) The only potential dependency on other methods is his proposed new intermediate path that avoids creating a list of all keys, by using iterkeys if it exists. (I suggested using iteritems to avoid the lookups.) If iter* aren't implemented, the only harm is falling back to the existing fallback of "for k in keys():" ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 16:07 Message: Logged In: YES user_id=80475 Originator: NO I'm -1 on making ANY guarantees about which methods underlie others -- that would constitute new and everlasting guarantees about how mappings are implemented. Subclasses should explicity override/extend the methods withed changed behavior. If that proves non-trivial, then it is likely there should be a has-a relationship instead of an is-a relationship. Also, it is likely that the subclass will have Liskov substitutability violations. Either way, there is probably a design flaw. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 14:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 12:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Tue Dec 19 23:29:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 14:29:37 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 08:08 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 17:29 Message: Logged In: YES user_id=764593 Originator: NO FWIW, I'm not sure I agree on not specifying which methods call share implementation. If someone hooks __getitem__ but not get, that is just asking for bugs. (The implementation of get may -- but need not -- make its own call to __getitem__, and not everyone will make the same decision.) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 17:26 Message: Logged In: YES user_id=764593 Originator: NO As I understand it, the problem is that dict.update is assuming any dict subclass will use the same internal data representation. Restricting the fast path to exactly builtin dicts (not subclasses) fixes the bug, but makes the fallback more frequent. The existing fallback is to call keys(), then iterate over it, retrieving the value for each key. (keys is required for a "minimal mapping" as documented is UserDict, and a few other random places.) The only potential dependency on other methods is his proposed new intermediate path that avoids creating a list of all keys, by using iterkeys if it exists. (I suggested using iteritems to avoid the lookups.) If iter* aren't implemented, the only harm is falling back to the existing fallback of "for k in keys():" ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 16:07 Message: Logged In: YES user_id=80475 Originator: NO I'm -1 on making ANY guarantees about which methods underlie others -- that would constitute new and everlasting guarantees about how mappings are implemented. Subclasses should explicity override/extend the methods withed changed behavior. If that proves non-trivial, then it is likely there should be a has-a relationship instead of an is-a relationship. Also, it is likely that the subclass will have Liskov substitutability violations. Either way, there is probably a design flaw. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 14:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 12:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Wed Dec 20 00:00:04 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 15:00:04 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 08:08 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 18:00 Message: Logged In: YES user_id=80475 Originator: NO It is also asking for bugs if someone hooks __getitem__ and starts to make possibly invalid assumptions about what other changes occur implicitly. Also, this patch kills the performance of builtin subclasses. If I subclass dict to add a new method, it would suck to have the performance of all of the other methods drop percariously. This patch is somewhat overzealous. It encroaches on the terriority of UserDict.DictMixin which was specifically made for propagating new behaviors. It unnecessarily exposes implementation details. It introduces implicit behaviors that should be done through explicit overrides of methods whose behavior is supposed to change. And, it is on the top of a slippery slope that we don't want to go down (i.e. do we want to guarantee that list.append is implemented in terms of list.extend, etc). Python has no shortage of places where builtin subclasses make direct calls to the underlying C code -- this patch leads to a bottomless pit of changes that kill performance and make implicit side-effects the norm instead of the exception. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 17:29 Message: Logged In: YES user_id=764593 Originator: NO FWIW, I'm not sure I agree on not specifying which methods call share implementation. If someone hooks __getitem__ but not get, that is just asking for bugs. (The implementation of get may -- but need not -- make its own call to __getitem__, and not everyone will make the same decision.) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 17:26 Message: Logged In: YES user_id=764593 Originator: NO As I understand it, the problem is that dict.update is assuming any dict subclass will use the same internal data representation. Restricting the fast path to exactly builtin dicts (not subclasses) fixes the bug, but makes the fallback more frequent. The existing fallback is to call keys(), then iterate over it, retrieving the value for each key. (keys is required for a "minimal mapping" as documented is UserDict, and a few other random places.) The only potential dependency on other methods is his proposed new intermediate path that avoids creating a list of all keys, by using iterkeys if it exists. (I suggested using iteritems to avoid the lookups.) If iter* aren't implemented, the only harm is falling back to the existing fallback of "for k in keys():" ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 16:07 Message: Logged In: YES user_id=80475 Originator: NO I'm -1 on making ANY guarantees about which methods underlie others -- that would constitute new and everlasting guarantees about how mappings are implemented. Subclasses should explicity override/extend the methods withed changed behavior. If that proves non-trivial, then it is likely there should be a has-a relationship instead of an is-a relationship. Also, it is likely that the subclass will have Liskov substitutability violations. Either way, there is probably a design flaw. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 14:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 12:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Wed Dec 20 01:13:03 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 16:13:03 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 08:08 Message generated for change (Comment added) made by rhettinger You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- >Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 19:13 Message: Logged In: YES user_id=80475 Originator: NO Since update already supports (key, item) changes, I do not see that rationale in trying to expand the definition what is dict-like to include a try-this, then try-that approach. This is a little too ad-hoc for too little benefit. Also, I do not see the point of adding PyMapping_Iterkeys to the C API. It affords no advantage over its macro definition (the current one-way-to-do-it). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 18:00 Message: Logged In: YES user_id=80475 Originator: NO It is also asking for bugs if someone hooks __getitem__ and starts to make possibly invalid assumptions about what other changes occur implicitly. Also, this patch kills the performance of builtin subclasses. If I subclass dict to add a new method, it would suck to have the performance of all of the other methods drop percariously. This patch is somewhat overzealous. It encroaches on the terriority of UserDict.DictMixin which was specifically made for propagating new behaviors. It unnecessarily exposes implementation details. It introduces implicit behaviors that should be done through explicit overrides of methods whose behavior is supposed to change. And, it is on the top of a slippery slope that we don't want to go down (i.e. do we want to guarantee that list.append is implemented in terms of list.extend, etc). Python has no shortage of places where builtin subclasses make direct calls to the underlying C code -- this patch leads to a bottomless pit of changes that kill performance and make implicit side-effects the norm instead of the exception. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 17:29 Message: Logged In: YES user_id=764593 Originator: NO FWIW, I'm not sure I agree on not specifying which methods call share implementation. If someone hooks __getitem__ but not get, that is just asking for bugs. (The implementation of get may -- but need not -- make its own call to __getitem__, and not everyone will make the same decision.) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 17:26 Message: Logged In: YES user_id=764593 Originator: NO As I understand it, the problem is that dict.update is assuming any dict subclass will use the same internal data representation. Restricting the fast path to exactly builtin dicts (not subclasses) fixes the bug, but makes the fallback more frequent. The existing fallback is to call keys(), then iterate over it, retrieving the value for each key. (keys is required for a "minimal mapping" as documented is UserDict, and a few other random places.) The only potential dependency on other methods is his proposed new intermediate path that avoids creating a list of all keys, by using iterkeys if it exists. (I suggested using iteritems to avoid the lookups.) If iter* aren't implemented, the only harm is falling back to the existing fallback of "for k in keys():" ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 16:07 Message: Logged In: YES user_id=80475 Originator: NO I'm -1 on making ANY guarantees about which methods underlie others -- that would constitute new and everlasting guarantees about how mappings are implemented. Subclasses should explicity override/extend the methods withed changed behavior. If that proves non-trivial, then it is likely there should be a has-a relationship instead of an is-a relationship. Also, it is likely that the subclass will have Liskov substitutability violations. Either way, there is probably a design flaw. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 14:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 12:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Wed Dec 20 02:22:15 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 17:22:15 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 15:53 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 20:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 12:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 20:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Wed Dec 20 06:33:48 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 21:33:48 -0800 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 06:51 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Closed Resolution: Accepted Priority: 7 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-19 21:33 Message: Logged In: YES user_id=33168 Originator: NO Lars, could you also create a patch to test this? Thanks! ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 14:07 Message: Logged In: YES user_id=849994 Originator: NO Applied in rev. 53090. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-19 13:50 Message: Logged In: YES user_id=642936 Originator: YES Yes, please apply complete-with-headererror.diff to the trunk. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 12:55 Message: Logged In: YES user_id=849994 Originator: NO What's the status of this patch? Does it still have to be applied somewhere? ---------------------------------------------------------------------- Comment By: Thomas Waldmann (thomaswaldmann) Date: 2006-09-30 12:28 Message: Logged In: YES user_id=100649 Could someone please backport this to 2.4 branch? Currently, 2.4.4c0 is broken when you try to extract a file that happens to have a "/" at index 99 in its name (tarinfo.name ends with "/" in this case, but this is NOT the end of the complete (LONG) filename). It gets misinterpreted as directory in this case. MoinMoin build process is broken on 2.4.4c0 due to this bug. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-09 17:23 Message: Logged In: YES user_id=33168 Ah, I see. I checked in the patch to remove the debug message since it's dead code. We are in feature freeze for 2.5, so I've updated the patch, attached it, set the group to 2.6. We can add the patch after 2.5 is out in a month or so. Committed revision 50503. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-09 01:01 Message: Logged In: YES user_id=642936 I attached a tar with two diffs: only-tarfile.diff only removes the debug message. complete-with-headererror.diff removes the debug message and adds the HeaderError exception to TarInfo.frombuf() mentioned below. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-04 15:04 Message: Logged In: YES user_id=33168 Lars, I don't see the debug (_dbg) call. Which version are you using? I didn't see r46040 in Lib/tafile.py for either 2.5 or 2.4. Could you point me to file/line/version where there's a problem (or just make a patch)? Thanks. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-03 01:59 Message: Logged In: YES user_id=642936 I would like to emphasize on point (2) of my previous post, which should be resolved before 2.5 comes out. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-18 03:00 Message: Logged In: YES user_id=642936 On (1): agreed. On (2): There is still a debug message emitted for a bad checksum: In TarInfo.frombuf() at the bottom a ValueError is raised if they don't match and is passed on to TarFile.next() where it is put out as a debug message using the _dbg() method in the except clause. The debug message where it is now (r46040) is senseless because the try-block will be left when TarInfo.frombuf() fails . On (3): You're right, I attached a patch that adds another Exception HeaderError which is raised in TarInfo.frombuf() instead of ValueError in case of a bad header. I hope that is acceptable. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-17 23:11 Message: Logged In: YES user_id=849994 Jim: I agree with your comments and have committed an improved version. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-05-17 15:25 Message: Logged In: YES user_id=764593 (1) Why change the exception style? When raising an instance, the style guide (PEP-8, http:// www.python.org/dev/peps/pep-0008/) prefers to construct that instance; the older form is left over from String exceptions and will be removed in Python 3. I could understand leaving them as they were, but if you're going to change them to make them consistent, why not use the current format? (2) Why get rid of the debug messages (such as the checksum check) entirely? Guarding them in if self.debug, I would understand. (3) I wouldn't count on str(e) (where e is any ValueError instance) being as meaningful as the (current version's) ReadError("empty, unreadable or compressed file") ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 09:26 Message: Logged In: YES user_id=849994 Thanks for the patch, applied as rev. 45954. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 06:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Wed Dec 20 07:42:38 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 19 Dec 2006 22:42:38 -0800 Subject: [Patches] [ python-Patches-1619247 ] fix urllib to raise IOError correctly Message-ID: Patches item #1619247, was opened at 2006-12-20 01:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619247&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Dug Song (dugsong) Assigned to: Nobody/Anonymous (nobody) Summary: fix urllib to raise IOError correctly Initial Comment: urllib.URLopener's http_error_default() is supposed to raise IOError on failure, but does this with an invalid number of arguments, resulting in a TypeError: File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib.py", line 357, in http_error_default raise IOError('http error', errcode, errmsg, headers) TypeError: EnvironmentError expected at most 3 arguments, got 4 removing the extraneous "headers" argument, the raised exception works as intended: File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib.py", line 357, in http_error_default raise IOError, ('http error', errcode, errmsg) IOError: [Errno http error] 401: 'Unauthorized' ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619247&group_id=5470 From noreply at sourceforge.net Wed Dec 20 09:16:30 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 00:16:30 -0800 Subject: [Patches] [ python-Patches-1619247 ] fix urllib to raise IOError correctly Message-ID: Patches item #1619247, was opened at 2006-12-20 06:42 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619247&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Duplicate Priority: 5 Private: No Submitted By: Dug Song (dugsong) Assigned to: Nobody/Anonymous (nobody) Summary: fix urllib to raise IOError correctly Initial Comment: urllib.URLopener's http_error_default() is supposed to raise IOError on failure, but does this with an invalid number of arguments, resulting in a TypeError: File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib.py", line 357, in http_error_default raise IOError('http error', errcode, errmsg, headers) TypeError: EnvironmentError expected at most 3 arguments, got 4 removing the extraneous "headers" argument, the raised exception works as intended: File "/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/urllib.py", line 357, in http_error_default raise IOError, ('http error', errcode, errmsg) IOError: [Errno http error] 401: 'Unauthorized' ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-20 08:16 Message: Logged In: YES user_id=849994 Originator: NO Thanks for reporting, this was already fixed in response to bug #1566800. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619247&group_id=5470 From noreply at sourceforge.net Wed Dec 20 09:38:13 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 00:38:13 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 12:53 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 00:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 17:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 09:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 17:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Wed Dec 20 10:25:02 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 01:25:02 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 12:53 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 01:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 00:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 17:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 09:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 17:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Wed Dec 20 12:51:05 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 03:51:05 -0800 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 15:51 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Closed Resolution: Accepted Priority: 7 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-20 12:51 Message: Logged In: YES user_id=642936 Originator: YES Created a testcase (which revealed an inconsistency). Thanks for the tip! Attached is a patch against test_tarfile.py and tarfile.py. File Added: tests-for-1484695.diff ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 06:33 Message: Logged In: YES user_id=33168 Originator: NO Lars, could you also create a patch to test this? Thanks! ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 23:07 Message: Logged In: YES user_id=849994 Originator: NO Applied in rev. 53090. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-19 22:50 Message: Logged In: YES user_id=642936 Originator: YES Yes, please apply complete-with-headererror.diff to the trunk. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 21:55 Message: Logged In: YES user_id=849994 Originator: NO What's the status of this patch? Does it still have to be applied somewhere? ---------------------------------------------------------------------- Comment By: Thomas Waldmann (thomaswaldmann) Date: 2006-09-30 21:28 Message: Logged In: YES user_id=100649 Could someone please backport this to 2.4 branch? Currently, 2.4.4c0 is broken when you try to extract a file that happens to have a "/" at index 99 in its name (tarinfo.name ends with "/" in this case, but this is NOT the end of the complete (LONG) filename). It gets misinterpreted as directory in this case. MoinMoin build process is broken on 2.4.4c0 due to this bug. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-10 02:23 Message: Logged In: YES user_id=33168 Ah, I see. I checked in the patch to remove the debug message since it's dead code. We are in feature freeze for 2.5, so I've updated the patch, attached it, set the group to 2.6. We can add the patch after 2.5 is out in a month or so. Committed revision 50503. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-09 10:01 Message: Logged In: YES user_id=642936 I attached a tar with two diffs: only-tarfile.diff only removes the debug message. complete-with-headererror.diff removes the debug message and adds the HeaderError exception to TarInfo.frombuf() mentioned below. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-05 00:04 Message: Logged In: YES user_id=33168 Lars, I don't see the debug (_dbg) call. Which version are you using? I didn't see r46040 in Lib/tafile.py for either 2.5 or 2.4. Could you point me to file/line/version where there's a problem (or just make a patch)? Thanks. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-03 10:59 Message: Logged In: YES user_id=642936 I would like to emphasize on point (2) of my previous post, which should be resolved before 2.5 comes out. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-18 12:00 Message: Logged In: YES user_id=642936 On (1): agreed. On (2): There is still a debug message emitted for a bad checksum: In TarInfo.frombuf() at the bottom a ValueError is raised if they don't match and is passed on to TarFile.next() where it is put out as a debug message using the _dbg() method in the except clause. The debug message where it is now (r46040) is senseless because the try-block will be left when TarInfo.frombuf() fails . On (3): You're right, I attached a patch that adds another Exception HeaderError which is raised in TarInfo.frombuf() instead of ValueError in case of a bad header. I hope that is acceptable. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-18 08:11 Message: Logged In: YES user_id=849994 Jim: I agree with your comments and have committed an improved version. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-05-18 00:25 Message: Logged In: YES user_id=764593 (1) Why change the exception style? When raising an instance, the style guide (PEP-8, http:// www.python.org/dev/peps/pep-0008/) prefers to construct that instance; the older form is left over from String exceptions and will be removed in Python 3. I could understand leaving them as they were, but if you're going to change them to make them consistent, why not use the current format? (2) Why get rid of the debug messages (such as the checksum check) entirely? Guarding them in if self.debug, I would understand. (3) I wouldn't count on str(e) (where e is any ValueError instance) being as meaningful as the (current version's) ReadError("empty, unreadable or compressed file") ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 18:26 Message: Logged In: YES user_id=849994 Thanks for the patch, applied as rev. 45954. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 15:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Wed Dec 20 12:55:40 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 03:55:40 -0800 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 13:51 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Closed Resolution: Accepted Priority: 7 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-12-20 11:55 Message: Logged In: YES user_id=849994 Originator: NO Applied as rev. 53106. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-20 11:51 Message: Logged In: YES user_id=642936 Originator: YES Created a testcase (which revealed an inconsistency). Thanks for the tip! Attached is a patch against test_tarfile.py and tarfile.py. File Added: tests-for-1484695.diff ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 05:33 Message: Logged In: YES user_id=33168 Originator: NO Lars, could you also create a patch to test this? Thanks! ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 22:07 Message: Logged In: YES user_id=849994 Originator: NO Applied in rev. 53090. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-19 21:50 Message: Logged In: YES user_id=642936 Originator: YES Yes, please apply complete-with-headererror.diff to the trunk. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-12-19 20:55 Message: Logged In: YES user_id=849994 Originator: NO What's the status of this patch? Does it still have to be applied somewhere? ---------------------------------------------------------------------- Comment By: Thomas Waldmann (thomaswaldmann) Date: 2006-09-30 19:28 Message: Logged In: YES user_id=100649 Could someone please backport this to 2.4 branch? Currently, 2.4.4c0 is broken when you try to extract a file that happens to have a "/" at index 99 in its name (tarinfo.name ends with "/" in this case, but this is NOT the end of the complete (LONG) filename). It gets misinterpreted as directory in this case. MoinMoin build process is broken on 2.4.4c0 due to this bug. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-10 00:23 Message: Logged In: YES user_id=33168 Ah, I see. I checked in the patch to remove the debug message since it's dead code. We are in feature freeze for 2.5, so I've updated the patch, attached it, set the group to 2.6. We can add the patch after 2.5 is out in a month or so. Committed revision 50503. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-09 08:01 Message: Logged In: YES user_id=642936 I attached a tar with two diffs: only-tarfile.diff only removes the debug message. complete-with-headererror.diff removes the debug message and adds the HeaderError exception to TarInfo.frombuf() mentioned below. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-07-04 22:04 Message: Logged In: YES user_id=33168 Lars, I don't see the debug (_dbg) call. Which version are you using? I didn't see r46040 in Lib/tafile.py for either 2.5 or 2.4. Could you point me to file/line/version where there's a problem (or just make a patch)? Thanks. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-07-03 08:59 Message: Logged In: YES user_id=642936 I would like to emphasize on point (2) of my previous post, which should be resolved before 2.5 comes out. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-18 10:00 Message: Logged In: YES user_id=642936 On (1): agreed. On (2): There is still a debug message emitted for a bad checksum: In TarInfo.frombuf() at the bottom a ValueError is raised if they don't match and is passed on to TarFile.next() where it is put out as a debug message using the _dbg() method in the except clause. The debug message where it is now (r46040) is senseless because the try-block will be left when TarInfo.frombuf() fails . On (3): You're right, I attached a patch that adds another Exception HeaderError which is raised in TarInfo.frombuf() instead of ValueError in case of a bad header. I hope that is acceptable. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-18 06:11 Message: Logged In: YES user_id=849994 Jim: I agree with your comments and have committed an improved version. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-05-17 22:25 Message: Logged In: YES user_id=764593 (1) Why change the exception style? When raising an instance, the style guide (PEP-8, http:// www.python.org/dev/peps/pep-0008/) prefers to construct that instance; the older form is left over from String exceptions and will be removed in Python 3. I could understand leaving them as they were, but if you're going to change them to make them consistent, why not use the current format? (2) Why get rid of the debug messages (such as the checksum check) entirely? Guarding them in if self.debug, I would understand. (3) I wouldn't count on str(e) (where e is any ValueError instance) being as meaningful as the (current version's) ReadError("empty, unreadable or compressed file") ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 16:26 Message: Logged In: YES user_id=849994 Thanks for the patch, applied as rev. 45954. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 13:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Wed Dec 20 13:59:18 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 04:59:18 -0800 Subject: [Patches] [ python-Patches-1615701 ] Creating dicts for dict subclasses Message-ID: Patches item #1615701, was opened at 2006-12-14 14:08 Message generated for change (Comment added) made by doerwalter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Walter D?rwald (doerwalter) Assigned to: Nobody/Anonymous (nobody) Summary: Creating dicts for dict subclasses Initial Comment: This patch changes dictobject.c so that creating dicts from mapping like objects only uses the internal dict functions if the argument is a *real* dict, not a subclass. This means that overwritten keys() and __getitem__() methods are now honored. In addition to that the fallback implementation now tries iterkeys() before trying keys(). It also adds a PyMapping_IterKeys() macro. ---------------------------------------------------------------------- >Comment By: Walter D?rwald (doerwalter) Date: 2006-12-20 13:59 Message: Logged In: YES user_id=89016 Originator: YES To clear up some apparent misunderstandings: This patch does *not* advocate that some dict methods should be implemented by calling other dict methods so that dict subclasses only have to overwrite a few methods to gain a completely consistent implementation. This patch only fixes the dict constructor (and update()) and consists of two parts: (1) There are three code paths in dictobject.c::dict_update_common(): (a) if the constructor argument doesn't have a "keys" attribute treat it as a iterable of items; (b) if the argument has a "keys" attribute, but is not a dict (and not an instance of a subclass of dict), use keys() and __getitem__() to make a copy of the mapping-like object. (c) if the argument has a "keys" attribute and is a dict (or an instance of a subclass of dict) bypass any of the overwritten methods that the object might provide and directly use the dict implementation. This patch changes PyDict_Merge() so that code path (b) is used for dict constructor arguments that are subclasses of dict, so that any overwritten methods are honored. (2) This means that now if a subclass of dict is passed to the constructor or update() the code is IMHO more correct (it honors the reimplemenation of the mapping methods), but slower. To reduce the slowdown instead of using kesY() and __getitem__(), iterkeys() and __getitem__() are used. I can't see why the current behaviour should be better: Yes, it is faster, but it is also wrong: Without the patch the behaviour of dict() and dict.update() depends on the fact whether the argument happens to subclass dict or not. If it doesn't all is well: the argument is treated as a mapping (i.e. keys() and __getitem__() are used) otherwise the methods are completely ignored. So can we agree on the fact that (1) is desirable? (At least Guido said as much: http://mail.python.org/pipermail/python-dev/2006-December/070341.html) BTW, I only added PyMapping_Iterkeys() because it mirrors PyMapping_Keys(). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-20 01:13 Message: Logged In: YES user_id=80475 Originator: NO Since update already supports (key, item) changes, I do not see that rationale in trying to expand the definition what is dict-like to include a try-this, then try-that approach. This is a little too ad-hoc for too little benefit. Also, I do not see the point of adding PyMapping_Iterkeys to the C API. It affords no advantage over its macro definition (the current one-way-to-do-it). ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-20 00:00 Message: Logged In: YES user_id=80475 Originator: NO It is also asking for bugs if someone hooks __getitem__ and starts to make possibly invalid assumptions about what other changes occur implicitly. Also, this patch kills the performance of builtin subclasses. If I subclass dict to add a new method, it would suck to have the performance of all of the other methods drop percariously. This patch is somewhat overzealous. It encroaches on the terriority of UserDict.DictMixin which was specifically made for propagating new behaviors. It unnecessarily exposes implementation details. It introduces implicit behaviors that should be done through explicit overrides of methods whose behavior is supposed to change. And, it is on the top of a slippery slope that we don't want to go down (i.e. do we want to guarantee that list.append is implemented in terms of list.extend, etc). Python has no shortage of places where builtin subclasses make direct calls to the underlying C code -- this patch leads to a bottomless pit of changes that kill performance and make implicit side-effects the norm instead of the exception. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 23:29 Message: Logged In: YES user_id=764593 Originator: NO FWIW, I'm not sure I agree on not specifying which methods call share implementation. If someone hooks __getitem__ but not get, that is just asking for bugs. (The implementation of get may -- but need not -- make its own call to __getitem__, and not everyone will make the same decision.) ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 23:26 Message: Logged In: YES user_id=764593 Originator: NO As I understand it, the problem is that dict.update is assuming any dict subclass will use the same internal data representation. Restricting the fast path to exactly builtin dicts (not subclasses) fixes the bug, but makes the fallback more frequent. The existing fallback is to call keys(), then iterate over it, retrieving the value for each key. (keys is required for a "minimal mapping" as documented is UserDict, and a few other random places.) The only potential dependency on other methods is his proposed new intermediate path that avoids creating a list of all keys, by using iterkeys if it exists. (I suggested using iteritems to avoid the lookups.) If iter* aren't implemented, the only harm is falling back to the existing fallback of "for k in keys():" ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2006-12-19 22:07 Message: Logged In: YES user_id=80475 Originator: NO I'm -1 on making ANY guarantees about which methods underlie others -- that would constitute new and everlasting guarantees about how mappings are implemented. Subclasses should explicity override/extend the methods withed changed behavior. If that proves non-trivial, then it is likely there should be a has-a relationship instead of an is-a relationship. Also, it is likely that the subclass will have Liskov substitutability violations. Either way, there is probably a design flaw. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-12-19 20:23 Message: Logged In: YES user_id=89016 Originator: YES iteritems() has to create a new tuple for each item, so this might be slower. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-19 18:50 Message: Logged In: YES user_id=764593 Originator: NO Why are you using iterkeys instead of iteritems? It seems like if they've filled out the interface enough to have iterkeys, they've probably filled it out all the way, and you do need the value as soon as you get the key. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1615701&group_id=5470 From noreply at sourceforge.net Wed Dec 20 19:04:52 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 10:04:52 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Wed Dec 20 19:43:00 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 10:43:00 -0800 Subject: [Patches] [ python-Patches-1610795 ] BSD version of ctypes.util.find_library Message-ID: Patches item #1610795, was opened at 2006-12-07 14:29 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Martin Kammerhofer (mkam) Assigned to: Thomas Heller (theller) Summary: BSD version of ctypes.util.find_library Initial Comment: The ctypes.util.find_library function for Posix systems is actually tailored for Linux systems. While the _findlib_gcc function relies only on the GNU compiler and may therefore work on any system with the "gcc" command in PATH, the _findLib_ld function relies on the /sbin/ldconfig command (originating from SunOS 4.0) which is not standardized. The version from GNU libc differs in option syntax and output format from other ldconfig programs around. I therefore provide a patch that enables find_library to properly communicate with the ldconfig program on FreeBSD systems. It has been tested on FreeBSD 4.11 and 6.2. It probably works on other *BSD systems too. (It works without this patch on FreeBSD, because after getting an error from ldconfig it falls back to _findlib_gcc.) While at it I also tidied up the Linux specific code: I'm escaping the function argument before interpolating it into a regular expression (to protect against nasty regexps) and removed the code for creation of a temporary file that was not used in any way. ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2006-12-20 19:43 Message: Logged In: YES user_id=11105 Originator: NO Unfortunately I'm unable to review or work on this patch *this year*. I will definitely take a look in January. Sorry. ---------------------------------------------------------------------- Comment By: Martin Kammerhofer (mkam) Date: 2006-12-12 12:28 Message: Logged In: YES user_id=1656067 Originator: YES Here is the revised patch. Tested on a (virtual) OpenBSD 3.9 machine, FreeBSD 4.11, FreeBSD 6.2 and DragonFlyBSD 1.6. Does not make assumptions on how many version numbers are appended to a library name any more. Even mixed length names (e.g. libfoo.so.8.9 vs. libfoo.so.10) compare in a meaningful way. (BTW: I also tried NetBSD 2.0.2, but its ldconfig is to different.) File Added: ctypes-util.py.patch ---------------------------------------------------------------------- Comment By: Martin Kammerhofer (mkam) Date: 2006-12-11 11:10 Message: Logged In: YES user_id=1656067 Originator: YES Hm, I did not know that OpenBSD is still using two version numbers for shared library. (I conclude that from the "libc.so.39.0" in the previous followup. Btw FreeBSD has used a MAJOR.MINOR[.DEWEY] scheme during the ancient days of the aout executable format.) Unfortunately my freebsd patch has the assumption of a single version number built in; more specifically the cmp(* map(lambda x: int(x.split('.')[-1]), (a, b))) is supposed to sort based an the last dot separated field. I guess that OpenBSD system does not have another libc, at least none with a minor > 0. ;-) Thomas, can you mail me the output of "ldconfig -r"? I will refine the patch then, doing a more general sort algorithm; i.e. sort by all trailing /(\.\d+)+/ fields. Said output from NetBSD welcome too. DragonflyBSD should be no problem since it is a fork of FreeBSD 4.8, but what looks its sys.platform like? ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-08 21:32 Message: Logged In: YES user_id=11105 Originator: NO I have tested the patch on FreeBSD 6.0 and (after extending the check to test for sys.platform.startswith("openbsd")) on OpenBSD 3.9 and it works fine. find_library("c") now returns libc.so.6 on FreeBSD 6.0, and libc.so.39.0 in OpenBSD 3.9, while it returned 'None' before on both machines. ---------------------------------------------------------------------- Comment By: David Remahl (chmod007) Date: 2006-12-08 08:50 Message: Logged In: YES user_id=2135 Originator: NO # Does this work (without the gcc fallback) on other *BSD systems too? I don't know, but it doesn't work on Darwin (which already has a custom method through macholib). ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2006-12-07 22:11 Message: Logged In: YES user_id=11105 Originator: NO Will do (although I would appreciate review from others too; I'm not exactly a BSD expert). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-07 20:15 Message: Logged In: YES user_id=21627 Originator: NO Thomas, can you take a look? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1610795&group_id=5470 From noreply at sourceforge.net Wed Dec 20 19:45:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 10:45:25 -0800 Subject: [Patches] [ python-Patches-1617699 ] slice-object support for ctypes Pointer/Array Message-ID: Patches item #1617699, was opened at 2006-12-18 05:28 Message generated for change (Comment added) made by theller You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617699&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Thomas Heller (theller) Summary: slice-object support for ctypes Pointer/Array Initial Comment: Support for slicing ctypes' Pointer and Array types with slice objects, although only for step=1 case. (Backported from p3yk-noslice branch.) ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2006-12-20 19:45 Message: Logged In: YES user_id=11105 Originator: NO Unfortunately I'm unable to review or work on this patch *this year*. I will definitely take a look in January. Sorry. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1617699&group_id=5470 From noreply at sourceforge.net Wed Dec 20 23:13:18 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 14:13:18 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 22:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 21 03:36:03 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 18:36:03 -0800 Subject: [Patches] [ python-Patches-1619846 ] Bug fixes for int unification branch Message-ID: Patches item #1619846, was opened at 2006-12-20 19:36 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Nobody/Anonymous (nobody) Summary: Bug fixes for int unification branch Initial Comment: This patch should fix all the real bugs in the int unification branch. All the remaining bugs are either external to the branch or due to tests that need updating (mostly due to the names of int vs long). External bugs: test_socket: http://sourceforge.net/tracker/index.php?func=detail&aid=1619659&group_id=5470&atid=105470 test_class: seems to be caused by using new-style classes by default. Unrelated to int-unification. test_set: inheritance of __hash__. I believe this was fixed in p3yk already. Test failures due to naming differences: test_ctypes test_doctest test_generators test_genexps test_optparse test_pyexpat Tests needing updating, not just due to name differences: test_descr test_pickletools The following aspects need specific review: PyLong_FromVoidPtr was doing the cast wrong. GCC was compiling the (unsigned Py_LONG_LONG)p cast in such a way as to produce a value larger than 2**32, obviously wrong on this 32bit box, and it warned about the cast too. Making it cast to Py_uintptr_t first seems to have corrected both the behaviour and the warning, but may be wrong on other architectures. Many of my changes to use PyInt_CheckExact may be better served by creating a PyInt_CheckSmall macro that retains the range check but allows subclasses. Alternatively, the index interface could be used, but would require more rewriting perhaps best left until later. There are some areas that handled signed vs unsigned and int vs long a bit differently, and they may still need work. Hard to tell what behaviour is correct in such cases. Skipped files: Doc/ext/run-func.c Mac/Modules/ctl/_Ctlmodule.c Mac/Modules/dlg/_Dlgmodule.c Mac/Modules/win/_Winmodule.c Mac/Modules/pycfbridge.c Modules/carbonevt/_CarbonEvtmodule.c Modules/_sqlite/connection.c Modules/almodule.c Modules/cgensupport.c Modules/clmodule.c Modules/flmodule.c Modules/grpmodule.c Modules/posicmodule.c:conv_confname Modules/pyexpat.c Modules/svmodule.c Modules/termios.c Modules/_bsddb.c Modules/_sqlite/statement.c PC/_winreg.c Python/dynload_beos.c Python/mactoolboxglue.c Python/marshal.c Python/pythonrun.c:handle_system_exit RISCOS/Modules/drawfmodule.c RISCOS/Modules/swimodule.c ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 From noreply at sourceforge.net Thu Dec 21 03:36:55 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 20 Dec 2006 18:36:55 -0800 Subject: [Patches] [ python-Patches-1619846 ] Bug fixes for int unification branch Message-ID: Patches item #1619846, was opened at 2006-12-20 19:36 Message generated for change (Settings changed) made by rhamphoryncus You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) >Assigned to: Guido van Rossum (gvanrossum) Summary: Bug fixes for int unification branch Initial Comment: This patch should fix all the real bugs in the int unification branch. All the remaining bugs are either external to the branch or due to tests that need updating (mostly due to the names of int vs long). External bugs: test_socket: http://sourceforge.net/tracker/index.php?func=detail&aid=1619659&group_id=5470&atid=105470 test_class: seems to be caused by using new-style classes by default. Unrelated to int-unification. test_set: inheritance of __hash__. I believe this was fixed in p3yk already. Test failures due to naming differences: test_ctypes test_doctest test_generators test_genexps test_optparse test_pyexpat Tests needing updating, not just due to name differences: test_descr test_pickletools The following aspects need specific review: PyLong_FromVoidPtr was doing the cast wrong. GCC was compiling the (unsigned Py_LONG_LONG)p cast in such a way as to produce a value larger than 2**32, obviously wrong on this 32bit box, and it warned about the cast too. Making it cast to Py_uintptr_t first seems to have corrected both the behaviour and the warning, but may be wrong on other architectures. Many of my changes to use PyInt_CheckExact may be better served by creating a PyInt_CheckSmall macro that retains the range check but allows subclasses. Alternatively, the index interface could be used, but would require more rewriting perhaps best left until later. There are some areas that handled signed vs unsigned and int vs long a bit differently, and they may still need work. Hard to tell what behaviour is correct in such cases. Skipped files: Doc/ext/run-func.c Mac/Modules/ctl/_Ctlmodule.c Mac/Modules/dlg/_Dlgmodule.c Mac/Modules/win/_Winmodule.c Mac/Modules/pycfbridge.c Modules/carbonevt/_CarbonEvtmodule.c Modules/_sqlite/connection.c Modules/almodule.c Modules/cgensupport.c Modules/clmodule.c Modules/flmodule.c Modules/grpmodule.c Modules/posicmodule.c:conv_confname Modules/pyexpat.c Modules/svmodule.c Modules/termios.c Modules/_bsddb.c Modules/_sqlite/statement.c PC/_winreg.c Python/dynload_beos.c Python/mactoolboxglue.c Python/marshal.c Python/pythonrun.c:handle_system_exit RISCOS/Modules/drawfmodule.c RISCOS/Modules/swimodule.c ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 From noreply at sourceforge.net Thu Dec 21 09:51:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 00:51:53 -0800 Subject: [Patches] [ python-Patches-1230446 ] tarfile.py: ExFileObject\'s tell() is wrong after readline() Message-ID: Patches item #1230446, was opened at 2005-06-30 18:23 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1230446&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) >Assigned to: Lars Gust?bel (gustaebel) >Summary: tarfile.py: ExFileObject\'s tell() is wrong after readline() Initial Comment: This patch is intended to be the solution to a problem that Niklas Volbers reported on comp.lang.python on June 1st. ExFileObject is the pseudo file object that is returned by TarFile.extractfile() and allows a user to read a file from the tar archive using a file interface. Niklas discovered that the tell() method didn't give correct results if he used it in conjunction with readline(). The cause for this is that readline() buffers the file data in 100 char blocks to be able to split it into lines. Thus, tell() always returns the file position in 100 byte steps. While I was looking for a fix to that problem in tarfile.py, I discovered that ExFileObject has another flaw as well: I read in readline()'s docstring that mixing calls to read() and readline() is not allowed, which is nowhere documented. I decided to put a little effort into a rewrite of ExFileObject. Here it is, tests included. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1230446&group_id=5470 From noreply at sourceforge.net Thu Dec 21 09:51:52 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 00:51:52 -0800 Subject: [Patches] [ python-Patches-1540385 ] tarfile __slots__ addition Message-ID: Patches item #1540385, was opened at 2006-08-15 06:47 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1540385&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Brian Harring (ferringb) >Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile __slots__ addition Initial Comment: Quicky mod, use __slots__ for TarInfo objects in tarfile module; lot of slots, but reduces memory by around 33% in testing. Pardon the delay in submitting; would like to see it in 2.5, but also well aware it's pushing it time wise. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-08-25 00:55 Message: Logged In: YES user_id=357491 I have backwards-compatibility issues with this. Adding slots to an already existing object can cause problems if people are arbitrarily adding attributes to an instance. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-08-16 15:28 Message: Logged In: YES user_id=642936 If you had run the test suite you would have noticed that there are still some names missing: buf, sparse and _link_target. Please adjust the patch. I cannot really estimate the implications of this patch. I think in terms of memory use it is reasonable to add __slots__ to Tarinfo objects because they can appear in hordes of thousands sometimes. But it could break some code, too. OTOH it is easy to reverse ;-) ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-08-15 08:55 Message: Logged In: YES user_id=21627 It definitely can't get into 2.5. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1540385&group_id=5470 From noreply at sourceforge.net Thu Dec 21 09:51:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 00:51:53 -0800 Subject: [Patches] [ python-Patches-1504073 ] Patch for 1496501 tarfile opener order Message-ID: Patches item #1504073, was opened at 2006-06-10 19:45 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1504073&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Jack Diederich (jackdied) >Assigned to: Lars Gust?bel (gustaebel) Summary: Patch for 1496501 tarfile opener order Initial Comment: when passing a fileobj into tarfile.open() without specifying the type open() will try openers in their dict key order from TarFile.OPEN_METH. This can fail if the order changes. This patch adds a tell() and seek() on failure of openers. fileobjs must already support these methods to work so this makes no new requirements. OpenOrderTest uses a dict-alike with shuffled keys to try and provoke the wrong orders for openers. Let me know if it looks good and I'll check it in. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1504073&group_id=5470 From noreply at sourceforge.net Thu Dec 21 09:51:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 00:51:53 -0800 Subject: [Patches] [ python-Patches-1262036 ] tarfile: fix for bug #1257255 Message-ID: Patches item #1262036, was opened at 2005-08-17 14:33 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1262036&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) >Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile: fix for bug #1257255 Initial Comment: (See bug #1257255 for a detailed description of the problem.) While the problem of the OP is that tarfile won't work when the current working dir has been removed from the filesystem, the more important one for me is that the test in the add() method, that should prevent the archive from being added to itself, will only succeed by accident. So, I decided to save the TarFile's name as an absolute path from the beginning to ensure that the archive cannot be added to itself even if the cwd changed during the operation. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-08-24 08:49 Message: Logged In: YES user_id=21627 The patch is actually incorrect. You cannot tar broken symlinks anymore, as os.path.samefile will raise an OSError (no such file or directory). Also, os.path.samefile doesn't exist on Windows. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-08-24 08:08 Message: Logged In: YES user_id=21627 Thanks for the patch. Committed as tarfile.py 1.30 NEWS 1.1339 tarfile.py 1.21.2.3 NEWS 1.1193.2.75 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1262036&group_id=5470 From noreply at sourceforge.net Thu Dec 21 09:51:52 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 00:51:52 -0800 Subject: [Patches] [ python-Patches-1507247 ] tarfile extraction does not honor umask Message-ID: Patches item #1507247, was opened at 2006-06-16 14:11 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Faik Uygur (faik) >Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile extraction does not honor umask Initial Comment: If the upperdirs in the member file's pathname does not exist. tarfile creates those paths with 0777 permission bits and does not honor umask. This patch uses umask to set the ti.mode of the created directory for later usage in chmod. --- tarfile.py (revision 46993) +++ tarfile.py (working copy) @@ -1560,7 +1560,9 @@ ti = TarInfo() ti.name = upperdirs ti.type = DIRTYPE - ti.mode = 0777 + umask = os.umask(0) + ti.mode = 0777 - umask + os.umask(umask) ti.mtime = tarinfo.mtime ti.uid = tarinfo.uid ti.gid = tarinfo.gid ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-06 00:40 Message: Logged In: YES user_id=161998 Originator: NO Hi, I can reproduce this problem on python 2.4 , and patch applies to python 2.5 too. Fix looks good to me. ---------------------------------------------------------------------- Comment By: Faik Uygur (faik) Date: 2006-08-18 11:44 Message: Logged In: YES user_id=1541018 Above patch is wrong. The correct one is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 From noreply at sourceforge.net Thu Dec 21 09:51:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 00:51:53 -0800 Subject: [Patches] [ python-Patches-1276378 ] tarfile: adding filed that use direct device addressing Message-ID: Patches item #1276378, was opened at 2005-08-30 09:50 Message generated for change (Settings changed) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1276378&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Urban Purkat (urbanp) >Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile: adding filed that use direct device addressing Initial Comment: The module tarfile has problems using os.stat() if the filenames use direct device addressing on Windows systems. For example: \\?\GLOBALROOT\Device\HarddiskVolume1\TEMP\1.txt The following error is reported: Traceback (most recent call last): File "C:\urbanp\py\test.py", line 9, in ? tar.add('\\\\?\\GLOBALROOT\\Device\\HarddiskVolume1\\TEMP\\1.txt', 'TEMP\\1.txt') File "C:\PROGRA~1\Python23\lib\tarfile.py", line 1204, in add tarinfo = self.gettarinfo(name, arcname) File "C:\PROGRA~1\Python23\lib\tarfile.py", line 1080, in gettarinfo statres = os.lstat(name) OSError: [Errno 2] No such file or directory: '\\\\?\\GLOBALROOT\\Device\\HarddiskVolume1\\TEMP\\1.txt' As a solution we can use win32file.GetFileAttributesEx(). I mapped win32file.GetFileAttributesEx result into a tuple that has the same structure as a os.stat result tuple. Therefore, the same variable can be used. The reason for using direct device addressing is that I need to create a tar package from a volume snapshot (vss on Windows 2003) that cannot be mounted. The solution also works on StorageCraft VSnap snapshots. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-09-01 00:13 Message: Logged In: YES user_id=1188172 I don't know whether direct device addressing is officially supported by any part of Python anyway. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1276378&group_id=5470 From noreply at sourceforge.net Thu Dec 21 15:49:51 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 06:49:51 -0800 Subject: [Patches] [ python-Patches-1620174 ] Improve platform.py usability on Windows Message-ID: Patches item #1620174, was opened at 2006-12-21 22:49 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Luke Dunstan (infidel) Assigned to: Nobody/Anonymous (nobody) Summary: Improve platform.py usability on Windows Initial Comment: This patch modifies platform.py to remove most of the dependencies on pywin32, and use the standard ctypes and _winreg modules instead. It also adds support for Windows CE. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470 From noreply at sourceforge.net Thu Dec 21 21:21:51 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 12:21:51 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 20:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 22:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 21 22:12:24 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 13:12:24 -0800 Subject: [Patches] [ python-Patches-1619846 ] Bug fixes for int unification branch Message-ID: Patches item #1619846, was opened at 2006-12-20 21:36 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) >Assigned to: Martin v. L?wis (loewis) Summary: Bug fixes for int unification branch Initial Comment: This patch should fix all the real bugs in the int unification branch. All the remaining bugs are either external to the branch or due to tests that need updating (mostly due to the names of int vs long). External bugs: test_socket: http://sourceforge.net/tracker/index.php?func=detail&aid=1619659&group_id=5470&atid=105470 test_class: seems to be caused by using new-style classes by default. Unrelated to int-unification. test_set: inheritance of __hash__. I believe this was fixed in p3yk already. Test failures due to naming differences: test_ctypes test_doctest test_generators test_genexps test_optparse test_pyexpat Tests needing updating, not just due to name differences: test_descr test_pickletools The following aspects need specific review: PyLong_FromVoidPtr was doing the cast wrong. GCC was compiling the (unsigned Py_LONG_LONG)p cast in such a way as to produce a value larger than 2**32, obviously wrong on this 32bit box, and it warned about the cast too. Making it cast to Py_uintptr_t first seems to have corrected both the behaviour and the warning, but may be wrong on other architectures. Many of my changes to use PyInt_CheckExact may be better served by creating a PyInt_CheckSmall macro that retains the range check but allows subclasses. Alternatively, the index interface could be used, but would require more rewriting perhaps best left until later. There are some areas that handled signed vs unsigned and int vs long a bit differently, and they may still need work. Hard to tell what behaviour is correct in such cases. Skipped files: Doc/ext/run-func.c Mac/Modules/ctl/_Ctlmodule.c Mac/Modules/dlg/_Dlgmodule.c Mac/Modules/win/_Winmodule.c Mac/Modules/pycfbridge.c Modules/carbonevt/_CarbonEvtmodule.c Modules/_sqlite/connection.c Modules/almodule.c Modules/cgensupport.c Modules/clmodule.c Modules/flmodule.c Modules/grpmodule.c Modules/posicmodule.c:conv_confname Modules/pyexpat.c Modules/svmodule.c Modules/termios.c Modules/_bsddb.c Modules/_sqlite/statement.c PC/_winreg.c Python/dynload_beos.c Python/mactoolboxglue.c Python/marshal.c Python/pythonrun.c:handle_system_exit RISCOS/Modules/drawfmodule.c RISCOS/Modules/swimodule.c ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-21 16:12 Message: Logged In: YES user_id=6380 Originator: NO Martin, do you have time to look at this? I'll play with it too but I'd like to have your opinion. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 From noreply at sourceforge.net Thu Dec 21 22:22:50 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 21 Dec 2006 13:22:50 -0800 Subject: [Patches] [ python-Patches-1619846 ] Bug fixes for int unification branch Message-ID: Patches item #1619846, was opened at 2006-12-21 03:36 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Adam Olsen (rhamphoryncus) Assigned to: Martin v. L?wis (loewis) Summary: Bug fixes for int unification branch Initial Comment: This patch should fix all the real bugs in the int unification branch. All the remaining bugs are either external to the branch or due to tests that need updating (mostly due to the names of int vs long). External bugs: test_socket: http://sourceforge.net/tracker/index.php?func=detail&aid=1619659&group_id=5470&atid=105470 test_class: seems to be caused by using new-style classes by default. Unrelated to int-unification. test_set: inheritance of __hash__. I believe this was fixed in p3yk already. Test failures due to naming differences: test_ctypes test_doctest test_generators test_genexps test_optparse test_pyexpat Tests needing updating, not just due to name differences: test_descr test_pickletools The following aspects need specific review: PyLong_FromVoidPtr was doing the cast wrong. GCC was compiling the (unsigned Py_LONG_LONG)p cast in such a way as to produce a value larger than 2**32, obviously wrong on this 32bit box, and it warned about the cast too. Making it cast to Py_uintptr_t first seems to have corrected both the behaviour and the warning, but may be wrong on other architectures. Many of my changes to use PyInt_CheckExact may be better served by creating a PyInt_CheckSmall macro that retains the range check but allows subclasses. Alternatively, the index interface could be used, but would require more rewriting perhaps best left until later. There are some areas that handled signed vs unsigned and int vs long a bit differently, and they may still need work. Hard to tell what behaviour is correct in such cases. Skipped files: Doc/ext/run-func.c Mac/Modules/ctl/_Ctlmodule.c Mac/Modules/dlg/_Dlgmodule.c Mac/Modules/win/_Winmodule.c Mac/Modules/pycfbridge.c Modules/carbonevt/_CarbonEvtmodule.c Modules/_sqlite/connection.c Modules/almodule.c Modules/cgensupport.c Modules/clmodule.c Modules/flmodule.c Modules/grpmodule.c Modules/posicmodule.c:conv_confname Modules/pyexpat.c Modules/svmodule.c Modules/termios.c Modules/_bsddb.c Modules/_sqlite/statement.c PC/_winreg.c Python/dynload_beos.c Python/mactoolboxglue.c Python/marshal.c Python/pythonrun.c:handle_system_exit RISCOS/Modules/drawfmodule.c RISCOS/Modules/swimodule.c ---------------------------------------------------------------------- >Comment By: Martin v. L?wis (loewis) Date: 2006-12-21 22:22 Message: Logged In: YES user_id=21627 Originator: NO Not this year anymore. I'll try to early next year (hopefully first week of January). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-21 22:12 Message: Logged In: YES user_id=6380 Originator: NO Martin, do you have time to look at this? I'll play with it too but I'd like to have your opinion. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1619846&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:00:08 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:00:08 -0800 Subject: [Patches] [ python-Patches-675976 ] mhlib does not obey MHCONTEXT env var Message-ID: Patches item #675976, was opened at 2003-01-28 04:16 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=675976&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) >Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Sjoerd Mullender (sjoerd) Assigned to: Nobody/Anonymous (nobody) Summary: mhlib does not obey MHCONTEXT env var Initial Comment: All programs in the (N)MH suite of programs use the MHCONTEXT environment variable to find the so-called context file where the current folder is remembered. mhlib should do the same, so that it can be used in combination with the standard (N)MH programs. Also, when writing the context file, mhlib should replace the Current-Folder line but keep the other lines in tact. The attached patch fixes both problems. It introduces a new method for the class MH called getcontextfile which uses the MHCONTEXT environment variable to find the context file, and it uses the already existing function updateline to update the context file. Some questions concerning this patch: - should I document the new method or should it be an internal method only? - should the fix be ported to older Python versions? With the patch it does behave differently if you have an MHCONTEXT environment variable. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:00 Message: Logged In: YES user_id=11375 Originator: NO The patch looks OK. Regarding your questions: 1) I think the method should be documented; it might be useful to subclasses of MH. 2) New feature, so 2.6 only. ---------------------------------------------------------------------- Comment By: Sjoerd Mullender (sjoerd) Date: 2003-02-13 04:48 Message: Logged In: YES user_id=43607 I can assure you that I did check that checkmark. Maybe it's my browser in combination with SF. We'll see if it works this time. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-02-12 22:02 Message: Logged In: YES user_id=33168 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=675976&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:09:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:09:42 -0800 Subject: [Patches] [ python-Patches-704676 ] add direct access to MD5 compression function to md5 module Message-ID: Patches item #704676, was opened at 2003-03-16 17:02 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=704676&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.3 >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: Reuben Sumner (rasumner) >Assigned to: A.M. Kuchling (akuchling) Summary: add direct access to MD5 compression function to md5 module Initial Comment: Access to the MD5 compression function allows doing things like calculating NMAC (see http://www.cs.ucsd.edu/users/mihir/papers/hmac.html). This patch gives such access. If accepted I am happy to do the same for SHA-1. I didn't update the doc strings very well, and I didn't update the documentation source at all. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:09 Message: Logged In: YES user_id=11375 Originator: NO Thanks for your contribution, but this seems quite specialized; NMAC doesn't appear to be widely used, perhaps because such API additions are required to implement it. The patch would also need to be updated for the introduction of hashlib in Python 2.5. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=704676&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:15:34 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:15:34 -0800 Subject: [Patches] [ python-Patches-703779 ] Fix strange warnings from interpreter if PYTHONSTARTUP used Message-ID: Patches item #703779, was opened at 2003-03-14 13:52 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=703779&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) >Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Stone (mbrierst) Assigned to: Nobody/Anonymous (nobody) Summary: Fix strange warnings from interpreter if PYTHONSTARTUP used Initial Comment: If the PYTHONSTARTUP environment variable is used, warnings will look strange. The warnings module uses __file__ to try to find the line causing the warning and print it out. __file__ is never reset after being set when processing the startup file. The attached patch fixes this by changing PyRun_SimpleFileExFlags to unset __file__ in the __main__ module dictionary entry after running a file, when it had been set by the routine in the first place. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:15 Message: Logged In: YES user_id=11375 Originator: NO The patch looks reasonable, and should be applied. I don't think this API change (no longer setting __file__) matters to callers of PyRun_SimpleFileExFlags ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-05-22 17:29 Message: Logged In: YES user_id=33168 Moving to patches ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=703779&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:25:36 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:25:36 -0800 Subject: [Patches] [ python-Patches-774221 ] 2.3c1: zipfile.py confused by garbage at the very end of zip Message-ID: Patches item #774221, was opened at 2003-07-19 11:00 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=774221&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Nikolai Saoukh (nms) >Assigned to: A.M. Kuchling (akuchling) Summary: 2.3c1: zipfile.py confused by garbage at the very end of zip Initial Comment: Some zip files can contain garbage past the End of Central Directory. _EndRecData() treat it as comment, thus failing sanity check for comment length. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:25 Message: Logged In: YES user_id=11375 Originator: NO Under what circumstances do zip files have garbage appended to them? Are there buggy zipfile-generating programs out there, or does the garbage come from, say, an extra newline added when a zipfile is attached to an e-mail? The patch looks OK, but I'm nervous about adding code to silently ignore an incorrect zipfile. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=774221&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:42:44 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:42:44 -0800 Subject: [Patches] [ python-Patches-783050 ] pty.fork() compatibility code wrong? Message-ID: Patches item #783050, was opened at 2003-08-04 16:02 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=783050&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: Library (Lib) Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Patrick Lynch (rvdp) >Assigned to: A.M. Kuchling (akuchling) Summary: pty.fork() compatibility code wrong? Initial Comment: pty.fork() provides compatibility code for environments lacking os.forkpty() This code is incorrect compared to the forkpty code in glibc, and will cause reading of the returned master_fd to hang indefinetly when the child process is finished. I am running on linux, Python 2.2. I am using the compatibility code where I need more control ( grabing stdout seperate to stderr, which is on the pty). To have it read like forkpty in glibc: <snip> master_fd, slave_fd = openpty() pid = os.fork() if pid == CHILD: # Establish a new session. os.setsid() os.close(master_fd) # Slave becomes stdin/stdout/stderr of child. os.dup2(slave_fd, STDIN_FILENO) os.dup2(slave_fd, STDOUT_FILENO) os.dup2(slave_fd, STDERR_FILENO) if (slave_fd > STDERR_FILENO): os.close (slave_fd) else:#PARENT os.close( slave_fd) # Parent and child process. return pid, master_fd Reading the master_fd when the child is finished will now raise an IO error, same as the native forkpty(). I guess the story may be different on non linux boxen Patrick. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:42 Message: Logged In: YES user_id=11375 Originator: NO Applied to the trunk in rev. 53146. Thanks for pointing out this bug! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=783050&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:46:40 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:46:40 -0800 Subject: [Patches] [ python-Patches-738094 ] for i in range(N) optimization Message-ID: Patches item #738094, was opened at 2003-05-15 03:14 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=738094&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.3 Status: Open Resolution: Later Priority: 2 Private: No Submitted By: Sebastien Keim (s_keim) Assigned to: Guido van Rossum (gvanrossum) Summary: for i in range(N) optimization Initial Comment: This patch is intended to special case the built-in range function in the common "for i in range(...):" construct. The goal is to make range() return an iterator instead of creating a real list, and then being able to depreciate the xrange type. It has once been suggested to make the compiler aware of the "for i in range(N):" construct and to make it able to produce optimized bytecode. But this solution is really hard to achieve because you have to ensure that the range built-in is not overridden. The patch take an opposite approach: it let the range built-in function looks at its execution context, and return an iterator if the next frame opcode to be executed is the GET_ITER opcode. Speed increase for the piece of code "for i in range(N): pass" : N (speed gain) 10 (+ 64%) 100 (+ 29%) 1000 (+ 23%) 10000 (+ 68%) 100000 (+108%) Since the patch only affect a small construct of the language, performance improvements for real applications are less impressive but they are still interesting: pystone.py (+7%) test_userstring.py (+8%) test_datetime.py (+20%) Note that the performance loss for "A = range(10)" is not measurable (less than 1%). If the patch is accepted, the same recipe may be applicable in some few other places. So the Py_IsIterationContext function must probably live somewhere else (is there a standard location for byte-code dependent stuff?). Maybe other opcodes (for sample JUMP_IF_FALSE) could provide other useful specialization contexts. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:46 Message: Logged In: YES user_id=11375 Originator: NO Should this patch simply be closed? Python 3.0 aims at fixing the range/xrange divide, doesn't it? This makes optimizations for 2.x not likely to be of great interest. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-03-03 09:48 Message: Logged In: YES user_id=1188172 I don't know if I fully understand, but doesn't it suffice to just use xrange()? ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-11 06:07 Message: Logged In: YES user_id=4771 Here is a safer patch. It adds a keyword argument 'iter' to range(), e.g.: >>> range(10, iter=True) and using an appropriate METH_XXX flag, the CALL_FUNCTION opcode now inserts a 'iter=True' keyword to the call when it is followed by GET_ITER. The patch doesn't live up to its performance promizes. I don't get any improvement at all on any real application. The only example it accelerates is a set of three nested loops :-( I still attach it for reference, and if someone else want to play with it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2003-07-09 12:22 Message: Logged In: YES user_id=6380 In the sake of stability for Python 2.3's accelerated release schedule, I'm postponing this until after 2.3. I'm also skeptical that it ca be absolutely correct. What if there is Python code of the form for i in some_function(): ... where some_function() is a C extension that at some point invokes range(), directly from C. Then when range() peeks in the opcode stream, it would believe that it was being called in the place of some_function(). So maybe I should just reject it as unsafe? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2003-05-18 17:18 Message: Logged In: YES user_id=6380 I'm interested, but have to ponder more, which will have to wait until I'm back from vacation. I expect that any hope to deprecate xrange() will prove naive -- people will want to pass ranges around between functions or reuse them (e.g. this happens a lot in timing tests). Maybe in Python 3.0 I can make range() act as an iterator generator. You'd have to say list(range(N)) to get an actual list then. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 19:25 Message: Logged In: YES user_id=80475 Assigning to Guido to see whether he is interested because it makes xrange less necessary or whether he thinks it is a horrendous hack --or maybe both ;-) ---------------------------------------------------------------------- Comment By: Sebastien Keim (s_keim) Date: 2003-05-15 11:14 Message: Logged In: YES user_id=498191 I have also thought about slicing, map and filter which could all be replaced by itertools equivalents , but I have failed to find a way to ensure that the argument lists aren't mutated during the for loop. Maybe it could be interesting to investigate into copy on write semantic for lists objects? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-15 10:33 Message: Logged In: YES user_id=80475 zip() would benefit greatly from your approach. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=738094&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:49:52 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:49:52 -0800 Subject: [Patches] [ python-Patches-783188 ] support for server side transactions in _ssl Message-ID: Patches item #783188, was opened at 2003-08-04 20:18 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=783188&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.3 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ilario Nardinocchi (illo) >Assigned to: A.M. Kuchling (akuchling) Summary: support for server side transactions in _ssl Initial Comment: Added a parameter (server_mode, type int) to the newPySSLObject() function. If its value is non-zero, a server SSL transaction is attempted - by using SSL_accept() instead of SSL_connect(). Also, the parameter has been added to the PySocket_ssl() constructor function (it defaults to 0 for compatibility) and the ssl() documentation reflects the change. A new python function has been added (pending(), C function PySSL_SSLpending()), which calls SSL_pending() and returns its return value as a python integer. Added documentation for this one, too. The return value is the number of bytes still to be read from the input buffer, so that if it's greater than zero it is useless to call a select() in a nonblocking scenery. Trivial patch but it's the only way to code an SSL server without third-party stuff. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-10-18 18:16 Message: Logged In: YES user_id=21627 Can you please provide patches for Doc/lib/libsocket.tex as well? Contribution of a small example of an SSL server in Demo/sockets would be appreciated as well. The demo should ship with a pre-generated self-signed certificate (private key without pass phrase); comments inside the sslserver.py should list the openssl commands used to generate the certificate. Are you interested in contributing such an example? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-08-05 02:28 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=783188&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:50:06 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:50:06 -0800 Subject: [Patches] [ python-Patches-783188 ] support for server side transactions in _ssl Message-ID: Patches item #783188, was opened at 2003-08-04 20:18 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=783188&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules >Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Ilario Nardinocchi (illo) Assigned to: A.M. Kuchling (akuchling) Summary: support for server side transactions in _ssl Initial Comment: Added a parameter (server_mode, type int) to the newPySSLObject() function. If its value is non-zero, a server SSL transaction is attempted - by using SSL_accept() instead of SSL_connect(). Also, the parameter has been added to the PySocket_ssl() constructor function (it defaults to 0 for compatibility) and the ssl() documentation reflects the change. A new python function has been added (pending(), C function PySSL_SSLpending()), which calls SSL_pending() and returns its return value as a python integer. Added documentation for this one, too. The return value is the number of bytes still to be read from the input buffer, so that if it's greater than zero it is useless to call a select() in a nonblocking scenery. Trivial patch but it's the only way to code an SSL server without third-party stuff. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-10-18 18:16 Message: Logged In: YES user_id=21627 Can you please provide patches for Doc/lib/libsocket.tex as well? Contribution of a small example of an SSL server in Demo/sockets would be appreciated as well. The demo should ship with a pre-generated self-signed certificate (private key without pass phrase); comments inside the sslserver.py should list the openssl commands used to generate the certificate. Are you interested in contributing such an example? ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-08-05 02:28 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=783188&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:50:50 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:50:50 -0800 Subject: [Patches] [ python-Patches-708374 ] add offset to mmap Message-ID: Patches item #708374, was opened at 2003-03-23 09:33 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=708374&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules >Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Neal Norwitz (nnorwitz) >Assigned to: A.M. Kuchling (akuchling) Summary: add offset to mmap Initial Comment: This patch is from Yotam Medini <yotamm at mellanox.co.il> sent to me in mail. It adds support for the offset parameter to mmap. It ignores the check for mmap size "if the file is character device. Some device drivers (which I happen to use) have zero size in fstat buffer, but still one can seek() read() and tell()." I added minimal doc and tests. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2006-10-28 17:54 Message: Logged In: YES user_id=341410 Right. The question that Martin had was if you have some mmap that has been mapped some offset into the large file, and you do mmap.find(...), do you return an offset relative to the file on disk, or relative to the mmap? So if I have m = mmap.mmap(..., startoffset=1024, length=1024), and I do m.find('X'), do I get some value -1...1023 inclusive (-1 for not found), or do I get some value 1024...2047 or -1? I think it only makes sense to return -1...1023, so that the return for .find() and other operations are relative to the mmap, not the file. ---------------------------------------------------------------------- Comment By: Yotam Medini (yotam) Date: 2006-10-28 17:46 Message: Logged In: YES user_id=22624 I am not sure I am following all arguments. But a major motivation for having an offset parameter, is to allow for mapping a segment of a huge file. A file could have a size which is too large for memory mapping. But one may need to map just a 'small' segment of a well known offset address. By the way, my experience in this case was in Linux only. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2006-10-28 14:29 Message: Logged In: YES user_id=341410 I agree. In the majority of use-cases that I would consider, offset into the current mmap object would be more useful than into the file. If the mmap has information about its offset in the file, then the user could easily get the file offset. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-10-28 14:12 Message: Logged In: YES user_id=21627 I have update the patch (mmap3.diff) for the current trunk (52498). I've reviewed and tested the Windows implementation, and found the original patch to be incorrect: For MapViewOfFile, instead of passing the size, offset+size must be passed. I also extended it to support 64-bit offsets on a 64-bit system (64-bit offsets on a 32-bit system still aren't supported). I have doubts about the changes to find and tell: why is it good that it includes the offset? In particular, for find, I think it should return the index in the buffer, not the offset in the file. ---------------------------------------------------------------------- Comment By: Yotam Medini (yotam) Date: 2003-12-08 14:01 Message: Logged In: YES user_id=22624 Recent posts deal mainly about with size checks against stat() call. But the main issue here is the support for offset in mmap()ping. I wonder what is the status of this so it would become part of the official release. If nobody cares to implement and test it for MS-Windows, let's have it for Linux/Unix only. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2003-07-15 09:07 Message: Logged In: YES user_id=11375 The device driver check is now committed to CVS, both the trunk and 2.2 maintenance branch. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2003-07-14 16:02 Message: Logged In: YES user_id=11375 According to my reading of the Single Unix Specification page for sys/stat.h, st_size only has a sensible value for regular files and for symlinks. This implies that the size comparison should only be done if S_ISREG() returns true. Patch attached as mmap_reg.diff; I'll also check it in after running the tests. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2003-07-11 09:54 Message: Logged In: YES user_id=11375 There's nothing to test on Windows; the HAVE_FSTAT code is inside #ifdef UNIX. If it works on Unix (and it seems to for me, on Linux), then it should be checked in. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-06-26 23:19 Message: Logged In: YES user_id=33168 Yes, the check for block devices should go in now as a bug fix I think. Can anyone test on windows? I attached a patch for just this minimal change. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2003-06-26 13:05 Message: Logged In: YES user_id=11375 Shouldn't block devices also be excluded from the size check? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-06-05 10:59 Message: Logged In: YES user_id=33168 Since this is an enhancement, it will not go into 2.2.x. The patch is stuck right now. It needs to be tested on Windows, but I can't do that (no Windows development environment). If you can update the patch to work on both Unix and Windows, we can apply the patch. ---------------------------------------------------------------------- Comment By: Yotam Medini (yotam) Date: 2003-06-05 10:55 Message: Logged In: YES user_id=22624 Just wondering, what is the status of this patch. I guess it did not make it to 2.2.3. regards -- yotam ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-04-22 23:53 Message: Logged In: YES user_id=80475 This is much closer and solves the last problem. But, it fails test_mmap with a windows errocode 8: not enought memory. Raymond ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-04-10 14:20 Message: Logged In: YES user_id=33168 http://msdn.microsoft.com/library/en-us/fileio/base/mapviewoffile.asp?frame=true says the offset must be a multiple of the allocation granularity which used to be hard-coded at 64k. So maybe if we made the offset 64k it would work. The attached patch makes this test change for windows only. (I think the Unix offset must be a multiple of PAGESIZE.) ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-04-10 13:53 Message: Logged In: YES user_id=80475 I had posted the wrong traceback (before the rebuild). The correct one shows Window Error #87. Traceback (most recent call last): File "test_mmap.py", line 357, in ? test_main() File "test_mmap.py", line 353, in test_main test_offset() File "test_mmap.py", line 37, in test_offset m = mmap.mmap(f.fileno(), mapsize - PAGESIZE, offset=PAGESIZE) WindowsError: [Errno 87] The parameter is incorrect [9363 refs] ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-04-10 13:14 Message: Logged In: YES user_id=33168 Note to self: Self, make sure to backport S_ISCHR() fix. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-04-10 13:10 Message: Logged In: YES user_id=33168 Hmmm, did Modules/mmapmodule.c get rebuilt? There is code in the patch for new_mmap_object() to support the offset keyword in both Unix & Windows. I don't see a problem in the code/patch. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-04-10 12:01 Message: Logged In: YES user_id=80475 It doesn't run: C:\py23\Lib\test>python_d test_mmap.py Traceback (most recent call last): File "test_mmap.py", line 357, in ? test_main() File "test_mmap.py", line 353, in test_main test_offset() File "test_mmap.py", line 37, in test_offset m = mmap.mmap(f.fileno(), mapsize - PAGESIZE, offset=PAGESIZE) TypeError: 'offset' is an invalid keyword argument for this function [9363 refs] ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-04-09 19:35 Message: Logged In: YES user_id=33168 Raymond, could you try to test this patch and see if it works on Windows? ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-03-29 16:28 Message: Logged In: YES user_id=33168 Sounds fair. Attached is an updated patch which includes windows support (I think). I cannot test on Windows. Tested on Linux. Includes updates for doc, src, and test. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-03-27 19:12 Message: Logged In: YES user_id=21627 I think non-zero offsets need to be supported for Windows as well. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2003-03-23 10:37 Message: Logged In: YES user_id=33168 Email received from Yotam: I have downloaded and patched the 2.3a source. compiled locally just this module, and it worked fine for my application (with offset for character device file) I did not run the released test though. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=708374&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:52:16 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:52:16 -0800 Subject: [Patches] [ python-Patches-793070 ] Add --remove-source option to setup.py Message-ID: Patches item #793070, was opened at 2003-08-22 06:10 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=793070&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Distutils and setup.py Group: None >Status: Closed >Resolution: Rejected Priority: 5 Private: No Submitted By: David Fraser (davidfraser) Assigned to: Nobody/Anonymous (nobody) Summary: Add --remove-source option to setup.py Initial Comment: For distributing non-opensource software, it is helpful to just distribute the .pyc/.pyo files and not the original .py files. The reverse (just distributing .py files) is possible through the --no-target-compile and --no-target-optimize switches to the distutils bdist command. We have added a --remove-source option which goes through and deletes all the source files from the build directory. This has been tested and works smoothly with Python 2.2.3 and seems to apply cleanly to Python 2.3 ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:52 Message: Logged In: YES user_id=11375 Originator: NO Closing this patch as requested by the original submitter. ---------------------------------------------------------------------- Comment By: David Fraser (davidfraser) Date: 2005-01-18 04:36 Message: Logged In: YES user_id=221678 I have now extracted the remove source code into a separate module, that can insert the required code into distutils and so be used without patching distutils. I'll attach it here for future reference. This can probably marked as closed since it is doable from outside ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2004-08-18 08:38 Message: Logged In: YES user_id=21627 The patch is still incorrect, though: it unconditionally decides that bdist_wininst is now going to ship byte code. This is incorrect, since the byte code can only belong to the current Python version, which might be different from the target Python version. In addition, "removing all source" means to remove all .py files. Instead, it should remove the files that are byte-compiled, i.e. the files returned from get_outputs(include_bytecode=0). ---------------------------------------------------------------------- Comment By: David Fraser (davidfraser) Date: 2003-09-01 12:25 Message: Logged In: YES user_id=221678 As far as I understand, the compiled files are generated in the build dir from the source files which have been copied there. Simply removing the source files afterwards means we don't have to make any changes to the whole way distutils operates. Also, we don't want to compile in-tree and then move the files as that might create confusing results when compiling for multiple versions of Python. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-08-31 12:18 Message: Logged In: YES user_id=21627 Wouldn't it be better not to copy the files into the build dir in the first place? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=793070&group_id=5470 From noreply at sourceforge.net Fri Dec 22 19:58:12 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 10:58:12 -0800 Subject: [Patches] [ python-Patches-813436 ] Scalable zipfile extension Message-ID: Patches item #813436, was opened at 2003-09-27 04:09 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=813436&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Marc De Falco (deufeufeu) Assigned to: Nobody/Anonymous (nobody) Summary: Scalable zipfile extension Initial Comment: Playing around with large zipfiles (> 10000 files), I've encountered big loading time, even if after having loaded it I use only 30 files in it. So I've introduced a differed parameter to the Zipfile.__init__ in order to load headers on-demand. As it's not a really good idea to activated it for all zip it defaults to False. I've updated the documentation too. Thx and keep the good work ;) P.S. : Dunno if it can be added to 2.3 or have to be included in 2.4, so I've choosed 2.4 group. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:58 Message: Logged In: YES user_id=11375 Originator: NO According to http://mail.python.org/pipermail/python-dev/2006-November/069969.html, the author of the zipfile rewrite isn't quite happy with it. It doesn't look like the new module will be API-compatible with zipfile, so I think this patch should still be considered for inclusion. ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2006-05-27 14:22 Message: Logged In: YES user_id=580910 Patch [1446489 ] zipfile: support for ZIP64 also addresses this as a side- effect of adding support ZIP64 support (for very big zipfiles). BTW. I don't quite understand why this patch is put on hold just because a rewrite of the zipfile module is planned. W.r.t. this patch: why is the on-demand loading optional? Loading the per-file headers when the zipfile is opened is not necessary for normal operation, the current zipfile module is basically doing a full verify of the zipfile on all occassions. This isn't necessary for normal operation and I don't think the infozip tools do this (probably because verification is very expensive). ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2006-05-25 10:38 Message: Logged In: YES user_id=81797 Actually, we'll leave it open until the Summer of Code implementation is completed and accepted. Sean ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2006-05-25 10:36 Message: Logged In: YES user_id=81797 There is a summer of code project to re-write the zipfile module, so this patch is moot. Sean ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=813436&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:00:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:00:07 -0800 Subject: [Patches] [ python-Patches-813436 ] Scalable zipfile extension Message-ID: Patches item #813436, was opened at 2003-09-27 04:09 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=813436&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Marc De Falco (deufeufeu) Assigned to: Nobody/Anonymous (nobody) Summary: Scalable zipfile extension Initial Comment: Playing around with large zipfiles (> 10000 files), I've encountered big loading time, even if after having loaded it I use only 30 files in it. So I've introduced a differed parameter to the Zipfile.__init__ in order to load headers on-demand. As it's not a really good idea to activated it for all zip it defaults to False. I've updated the documentation too. Thx and keep the good work ;) P.S. : Dunno if it can be added to 2.3 or have to be included in 2.4, so I've choosed 2.4 group. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 14:00 Message: Logged In: YES user_id=11375 Originator: NO Patch #1446489, mentioned in Ronald's 2006-05-27 14:22 comment, was committed and is in Python 2.5. Is this patch still relevant? ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:58 Message: Logged In: YES user_id=11375 Originator: NO According to http://mail.python.org/pipermail/python-dev/2006-November/069969.html, the author of the zipfile rewrite isn't quite happy with it. It doesn't look like the new module will be API-compatible with zipfile, so I think this patch should still be considered for inclusion. ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2006-05-27 14:22 Message: Logged In: YES user_id=580910 Patch [1446489 ] zipfile: support for ZIP64 also addresses this as a side- effect of adding support ZIP64 support (for very big zipfiles). BTW. I don't quite understand why this patch is put on hold just because a rewrite of the zipfile module is planned. W.r.t. this patch: why is the on-demand loading optional? Loading the per-file headers when the zipfile is opened is not necessary for normal operation, the current zipfile module is basically doing a full verify of the zipfile on all occassions. This isn't necessary for normal operation and I don't think the infozip tools do this (probably because verification is very expensive). ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2006-05-25 10:38 Message: Logged In: YES user_id=81797 Actually, we'll leave it open until the Summer of Code implementation is completed and accepted. Sean ---------------------------------------------------------------------- Comment By: Sean Reifschneider (jafo) Date: 2006-05-25 10:36 Message: Logged In: YES user_id=81797 There is a summer of code project to re-write the zipfile module, so this patch is moot. Sean ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=813436&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:01:26 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:01:26 -0800 Subject: [Patches] [ python-Patches-742598 ] SocketServer timeout, zombies Message-ID: Patches item #742598, was opened at 2003-05-23 17:11 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=742598&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) >Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Pomraning (pilcrow) >Assigned to: A.M. Kuchling (akuchling) Summary: SocketServer timeout, zombies Initial Comment: Patch adds class variable 'timeout' to BaseServer, and methods await_request and handle_timeout. Backwards compatible API -- timeout is None by default, meaning await_request jumps directly to get_request and never calls handle_timeout. Facilitates periodic server bookkeeping (check config files, intervalic logging, etc.), if you supply your own serve loop. Additionally, ForkingMixin sets timeout to 300 seconds, and overrides handle_timeout to call collect_children. Upshot: n zombies linger in process table until 5 minutes pass or new request arrives, whichever comes first. docstring updates, too (s/reuse_address/allow_reuse_address, etc.) -Mike ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=742598&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:09:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:09:07 -0800 Subject: [Patches] [ python-Patches-827559 ] SimpleHTTPServer directory-indexing " bug" fix Message-ID: Patches item #827559, was opened at 2003-10-21 10:49 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=827559&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) >Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Chris Gonnerman (solomoriah) >Assigned to: A.M. Kuchling (akuchling) Summary: SimpleHTTPServer directory-indexing "bug" fix Initial Comment: The SimpleHTTPServer.py module, when presented with a URL of the form: http://some.site.com/directory should redirect the browser to: http://some.site.com/directory/ This is evidently a standard behavior of major webservers (Apache and IIS at least). SimpleHTTPServer does not do this, but fortunately it is simple to implement (patch attached, naturally). I am providing a fix for 2.2 but this problem appears to be in earlier versions, and is also in 2.3. This patch appears to work fine for 2.3. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 14:09 Message: Logged In: YES user_id=11375 Originator: NO Committed to the trunk in rev. 53147 and to 25-maint in rev. 53148. Thanks for the fix! ---------------------------------------------------------------------- Comment By: Titus Brown (titus) Date: 2004-12-19 03:20 Message: Logged In: YES user_id=23486 Clearly a bug, IMO: a reference to a directory name w/o a slash will return the directory listing, but relative links do not then work. (Does this need more explication or ???) Patch works against latest CVS & fixes behavior. Recommend application. ---------------------------------------------------------------------- Comment By: Chris Gonnerman (solomoriah) Date: 2003-11-14 00:28 Message: Logged In: YES user_id=321948 Oops. Allow me to clarify. Directory names without a trailing slash need to be redirected to the same name with the trailing slash to avoid breaking relative links from the page. I use SimpleHTTPServer to test websites which will be deployed via Apache. Relative links on default directory pages (index.html) are broken using SimpleHTTPServer but work correctly on Apache. This is a de facto standard at least (I haven't read the RFC's but both IE and Mozilla expect this behavior). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2003-11-13 10:54 Message: Logged In: YES user_id=21627 Why is this a bug? SimpleHTTPServer already returns the directory content if only the directory name is given, with or without trailing slash. I believe Apache does that only as an implementation detail, and it is unfortunate that this detail is exposed to clients. So unless you can report a specific problem with the approach taken by SimpleHTTPServer, I'm tempted to reject this patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=827559&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:24:46 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:24:46 -0800 Subject: [Patches] [ python-Patches-1045783 ] Remove unncessary NLST from ftp transfers Message-ID: Patches item #1045783, was opened at 2004-10-12 19:05 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1045783&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Chris Cogdon (chriscog) >Assigned to: A.M. Kuchling (akuchling) Summary: Remove unncessary NLST from ftp transfers Initial Comment: Using urllib (or urllib2) to open FTP URLs will attempt a 'nlst' (directory listing) before retrieving the file. Some FTP servers block directory listings, but allow 'getting' files just fine. Having urllib do a NLST before retriving files is totally unnecessary, and will fail given the FTP server mentioned above. If a file is not 'gettable', which is what the probable intention of NLST is, it will fail in the retrieve process. This patch removes the NLST from urllib. As a bonus, this makes FTP transfers faster. ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 14:24 Message: Logged In: YES user_id=11375 Originator: NO This fix seems to have been applied in rev. 45819. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-05-19 16:49 Message: Logged In: YES user_id=261020 Looks reasonable to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1045783&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:26:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:26:37 -0800 Subject: [Patches] [ python-Patches-1097797 ] Encoding for Code Page 273 used by EBCDIC Germany Austria Message-ID: Patches item #1097797, was opened at 2005-01-07 06:44 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1097797&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Bierenfeld (mbieren) Assigned to: Nobody/Anonymous (nobody) Summary: Encoding for Code Page 273 used by EBCDIC Germany Austria Initial Comment: CP273.TXT to be used with gencodec.py. Enables Communication with "old" IBM Mainframes ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 14:26 Message: Logged In: YES user_id=11375 Originator: NO Given that the patch author has provided further details, can this patch be accepted now? ---------------------------------------------------------------------- Comment By: Michael Bierenfeld (mbieren) Date: 2005-01-07 09:45 Message: Logged In: YES user_id=1074928 The encoding is used on IBM Mainframes here in Germany and Austria. Mostly on "old" VM/VSE Machines. Rawdata is transferred via MQSeries and then encoded from "cp273" to "latin_1". The file has been translated from the original IBM RFC by myself. www.unicode.org has no equivalent file on their FTP-Servers (ftp://ftp.unicode.org/Public/MAPPINGS/). CP273 is an alias for this charset. source: IBM NLS RM Vol2 SE09-8002-01, March 1990 Regards Michael ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2005-01-07 07:25 Message: Logged In: YES user_id=38388 Can you provide a reference of where and how this encoding is used ? We also need information of where the attached file originated. Thanks. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1097797&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:30:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:30:42 -0800 Subject: [Patches] [ python-Patches-1094542 ] add Bunch type to collections module Message-ID: Patches item #1094542, was opened at 2005-01-02 13:26 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1094542&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: Modules Group: Python 2.5 >Status: Closed >Resolution: Postponed Priority: 5 Private: No Submitted By: Steven Bethard (bediviere) Assigned to: Nobody/Anonymous (nobody) Summary: add Bunch type to collections module Initial Comment: This patch adds a proposed Bunch type to the collections module which complements Python's dict type, which provides a name-value mapping through the __getitem__ protocol, with the Bunch type, which provides an attribute-value mapping through dotted-attribute access. Example: >>> x = Bunch(spam=Bunch(rabbit=1, badger=[2, 3, 4]), ham='neewom') >>> x.spam.badger [2, 3, 4] >>> x.ham 'neewom' ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 14:30 Message: Logged In: YES user_id=11375 Originator: NO It looks like the PEP was never added to the PEP index. Do you want to resubmit it? I'll close this patch per the original submitter's request. ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-02-16 02:16 Message: Logged In: YES user_id=945502 For anyone still interested, development on this type has moved to: http://namespace.python-hosting.com/ The type has been renamed Namespace. I guess this patch should be withdrawn. We'll submit a better version (with an updated PEP) when more details have been hashed out, and we have a little more use data. ---------------------------------------------------------------------- Comment By: Alan Green (alanvgreen) Date: 2005-01-26 19:50 Message: Logged In: YES user_id=1174944 AttrMap and AttributeMapping aren't great names either, because a Bunch isn't like a Map. Here's two more suggestions: - Plain or PlainObject (implying that these are plain objects that the program can adorn in any way it likes) - Holder, AttributeHolder, DataHolder, or Data (If I were in a particularly whimsical mood, I might also suggest 'Eric', after the Monty Python song "Eric the half-a-bee", because a Bunch is only half-an-object - it only has data and no functions. But I'm not, so I won't.) ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-01-25 16:01 Message: Logged In: YES user_id=945502 Yes, I definitely agree. I only used Bunch because it was the least controversial (and was the name used in the Cookbook). Unfortunately, it's also too vague. I feel a little uneasy about Record or Struct because, to me, they seem to imply an ordering of the attributes, while the attributes of the type here are unordered. Struct also already has a meaning in the stdlib which is quite different. Namespace has the right meaning, but I think it's so overloaded for XML that this could be confusing. Perhaps something like AttributeMapping? AttrMap, maybe? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-01-25 15:42 Message: Logged In: YES user_id=80475 This needs a better name than Bunch. The clearly suggested a collection but is otherwise amorphous and fails to suggest any relevant properties. In language design, naming is vital. ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-01-25 15:06 Message: Logged In: YES user_id=945502 Ok, I'll post the current PEP draft below. Of course, I welcome any suggestions to make its acceptance more likely. I'll definitely update the __eq__ docstring as suggested, but I'll probably wait until I've heard what other changes should be made. ---------------------------------------------------------------------- PEP: XXX Title: Generic Object Data Type Version: $Revision: 1.0 $ Last-Modified: $Date: 2004/11/29 16:00:00 $ Author: Steven Bethard Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 29-Nov-2004 Python-Version: 2.5 Post-History: 29-Nov-2004 Abstract ======== This PEP proposes a standard library addition to support the simple creation of 'generic' objects which can be given named attributes without the need to declare a class. Such attribute-value mappings are intended to complement the name-value mappings provided by Python's builtin dict objects. Motivation ========== Python's dict objects provide a simple way of creating anonymous name-value mappings. These mappings use the __getitem__ protocol to access the value associated with a name, so that code generally appears like:: mapping['name'] Occasionally, a programmer may decide that dotted-attribute style access is more appropriate to the domain than __getitem__ style access, and that their mapping should be accessed like:: mapping.name Currently, if a Python programmer makes this design decision, they are forced to declare a new class, and then build instances of this class. When no methods are to be associated with the attribute-value mappings, declaring a new class can be overkill. This PEP proposes adding a simple type to the collections module of the standard library that can be used to build such attribute-value mappings. Providing such a type allows the Python programmer to determine which type of mapping is most appropriate to their domain and apply this choice with minimal effort. Some of the suggested uses include: Returning Named Results ----------------------- It is often appropriate for a function that returns multiple items to give names to the different items returned. The type suggested in this PEP provides a simple means of doing this that allows the returned values to be accessed in the usual attribute-style access:: >>> def f(x): ... return Bunch(double=2*x, squared=x**2) ... >>> y = f(10) >>> y.double 20 >>> y.squared 100 Representing Hierarchical Data ------------------------------ The type suggested in this PEP also allows a simple means of representing hierarchical data that allows attribute-style access:: >>> x = Bunch(spam=Bunch(rabbit=1, badger=[2, 3, 4]), ham='neewom') >>> x.spam.badger [2, 3, 4] >>> x.ham 'neewom' Rationale ========= As Bunch objects are intended primarily to replace simple, data-only classes, simple Bunch construction was a primary concern. As such, the Bunch constructor supports creation from keyword arguments, dicts, and sequences of (attribute, value) pairs:: >>> Bunch(eggs=1, spam=2, ham=3) Bunch(eggs=1, ham=3, spam=2) >>> Bunch({'eggs':1, 'spam':2, 'ham':3}) Bunch(eggs=1, ham=3, spam=2) >>> Bunch([('eggs',1), ('spam',2), ('ham',3)]) Bunch(eggs=1, ham=3, spam=2) To allow attribute-value mappings to be easily combined, the update method of Bunch objects supports similar arguments. If Bunch objects are used to represent hierarchical data, comparison of such objects becomes a concern. For this reason, Bunch objects support object equality:: >>> x = Bunch(parrot=Bunch(lumberjack=True, spam=42), peng='shrub') >>> y = Bunch(peng='shrub', parrot=Bunch(spam=42, lumberjack=True)) >>> z = Bunch(parrot=Bunch(lumberjack=True), peng='shrub') >>> x == y True >>> x == z False Note that support for the various mapping methods, e.g. __(get|set|del)item__, __len__, __iter__, __contains__, items, keys, values, etc. was intentionally omitted as these methods did not seem to be necessary for the core uses of an attribute-value mapping. If such methods are truly necessary for a given use case, this may suggest that a dict object is a more appropriate type for that use. Examples ========= Converting an XML DOM tree into a tree of nested Bunch objects:: >>> import xml.dom.minidom >>> def getbunch(element): ... result = Bunch() ... if element.attributes: ... result.update(element.attributes.items()) ... children = {} ... for child in element.childNodes: ... if child.nodeType == xml.dom.minidom.Node.TEXT_NODE: ... children.setdefault('text', []).append( ... child.nodeValue) ... else: ... children.setdefault(child.nodeName, []).append( ... getbunch(child)) ... result.update(children) ... return result ... >>> doc = xml.dom.minidom.parseString("""\ ... ... ... a text 1 ... ... b text ... a text 2 ... ... c text ... """) >>> b = getbunch(doc.documentElement) >>> b.a[0].b[1] Bunch(attr_b=u'3', text=[u' b text ']) Reference Implementation ======================== The code is available as SourceForge patch 1094542 [1]_. Open Issues =========== What should the type be named? Some suggestions include 'Bunch', 'Record', 'Struct' and 'Namespace'. Where should the type be placed? The current suggestion is the collections module. References ========== .. [1] http://sourceforge.net/tracker/index.php?func=detail&aid=1094542&group_id=5470&atid=305470 .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: ---------------------------------------------------------------------- Comment By: Alan Green (alanvgreen) Date: 2005-01-24 07:34 Message: Logged In: YES user_id=1174944 Some thoughts while we wait for the PEP: 1. This doesn't look like a collection class to me. It's more of a convenient substitute for a first-class object. The PEP would need to include a rationale as to why this class is in the collections module. 2. It may be desireable to make the core parts of Bunch (equality test, repr, and update) available as functions that other classes can use if appropriate. This might help developers building objects more complex than a Bunch. Then again, maybe not, but I'd like to see the PEP address this. 3. The docstring on __eq__ should explicitly say what consitutes equality for bunches: both bunches have the same attributes and the same values for those attributes. 4. It's easy enough to convert a dict to a Bunch (via the Bunch constructor), but I would have expected that there be a way to convert a Bunch to a dict. Overall, a useful concept, but I'd like to read the PEP - you could always upload your draft to this patch item :) ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-01-10 10:15 Message: Logged In: YES user_id=945502 I submitted a PEP for it on 2 Jan 2005, but I haven't heard back from peps at python.org yet. Sorry, I didn't realize it might take so long to get a PEP number. ---------------------------------------------------------------------- Comment By: Michael Chermside (mcherm) Date: 2005-01-10 09:08 Message: Logged In: YES user_id=99874 Would someone be willing to provide the motivation for adding this class? I'm certainly willing to listen, but I'm not convinced this is worth adding to the std library. (I guess that's a -0 vote.) ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-01-03 12:38 Message: Logged In: YES user_id=1188172 Let me add that the main functionality consists in the easy initializing and updating (otherwise, you just could use an empty class). I'm +1 on the class, but I would call it `bunch'. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1094542&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:31:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:31:42 -0800 Subject: [Patches] [ python-Patches-1097797 ] Encoding for Code Page 273 used by EBCDIC Germany Austria Message-ID: Patches item #1097797, was opened at 2005-01-07 12:44 Message generated for change (Comment added) made by lemburg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1097797&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Bierenfeld (mbieren) Assigned to: Nobody/Anonymous (nobody) Summary: Encoding for Code Page 273 used by EBCDIC Germany Austria Initial Comment: CP273.TXT to be used with gencodec.py. Enables Communication with "old" IBM Mainframes ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2006-12-22 20:31 Message: Logged In: YES user_id=38388 Originator: NO I guess so. Note that it's not a patch, but the template to be used with gencodec.py. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 20:26 Message: Logged In: YES user_id=11375 Originator: NO Given that the patch author has provided further details, can this patch be accepted now? ---------------------------------------------------------------------- Comment By: Michael Bierenfeld (mbieren) Date: 2005-01-07 15:45 Message: Logged In: YES user_id=1074928 The encoding is used on IBM Mainframes here in Germany and Austria. Mostly on "old" VM/VSE Machines. Rawdata is transferred via MQSeries and then encoded from "cp273" to "latin_1". The file has been translated from the original IBM RFC by myself. www.unicode.org has no equivalent file on their FTP-Servers (ftp://ftp.unicode.org/Public/MAPPINGS/). CP273 is an alias for this charset. source: IBM NLS RM Vol2 SE09-8002-01, March 1990 Regards Michael ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2005-01-07 13:25 Message: Logged In: YES user_id=38388 Can you provide a reference of where and how this encoding is used ? We also need information of where the attached file originated. Thanks. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1097797&group_id=5470 From noreply at sourceforge.net Fri Dec 22 20:33:30 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 11:33:30 -0800 Subject: [Patches] [ python-Patches-1182394 ] HMAC hexdigest and general review Message-ID: Patches item #1182394, was opened at 2005-04-13 13:08 Message generated for change (Settings changed) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1182394&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Shane Holloway (shane_holloway) >Assigned to: A.M. Kuchling (akuchling) Summary: HMAC hexdigest and general review Initial Comment: This patch is primarily aimed at speeding up the HMAC.hexdigest method. The current implementation is:: return "".join([hex(ord(x))[2:].zfill(2) for x in tuple(self.digest())]) Which is correct, but a little too clever. Especially when other tools are available. The attached patch adds HMAC._current that returns the hash object combined from self.inner and self.outter to be used by HMAC.digest and HMAC.hexdigest. This results in an 11 fold speed improvement for md5 and sha digestmod's while also making the code a little easier. Other review points for the HMAC module include removing "_secret_backdoor_key" in favor of using "None" as the sentinel. In this case, it makes sense to use None because it is an invalid key, avoids a global lookup in py2.4 and beyond, and is just as readable. I also moved blocksize to a class level constant. This allows implementations to create different blocksize hashes by inheriting and changing this value. This also involved a change to copy() to use self.__class__(None) instead of directly calling HMAC(None). The final comment is that it would be nice to have a C-level implementation of _strxor in binascii for this module and possibly other uses. However, I don't really know a significantly faster way of implementing this method in python. Thanks, -Shane Holloway ---------------------------------------------------------------------- Comment By: Shane Holloway (shane_holloway) Date: 2005-07-26 13:20 Message: Logged In: YES user_id=283742 Yes, I was reviewing HMAC to learn how it works, and to see if I could use it for a distributed verification algorithm. I will be make a fair number of hmac.copy() calls, but it will by no means be critical. Mainly, I found the implementation more than a little clunky to read, and significantly slower than it could be. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2005-06-16 14:08 Message: Logged In: YES user_id=11375 The HMAC._current() change and adding self.blocksize both look reasonable. I don't see the point of the _secret_backdoor_key change, though; accidentally using None or calling HMAC() will get an object that fails to work, and the speedup seems too small to compensate for this. Do you have a use case that requires making lots of HMAC.copy() calls? If the backdoor_key gets taken out, I don't think we need to keep the name around for backward compatibility; no callers of the module should have been using it. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-04-26 12:32 Message: Logged In: YES user_id=341410 Obviously "always have a length of 4" should have been "always have a length of zero modulo 4". ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-04-26 03:10 Message: Logged In: YES user_id=341410 I think the patch looks good. Though perhaps leaving a... _secret_backdoor_key = None ... for backwards compatibility is a good idea. The fastest method I've found (in pure Python) for xoring strings is to use an array.array, with the largest typecode that is a divisor of your sequence (cooking your strings to always have a length of 4 and to use signed 4 byte longs seems to work well on PII/PIII based systems). Alternatively, writing it in C and compiling it with the new GCC 4 will offer automatic vectorization, and save allocating Python integers. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-14 03:54 Message: Logged In: YES user_id=6656 Attempting to add the ^ for strings is a non-starter. No opinion on the rest of the patch. ---------------------------------------------------------------------- Comment By: Shane Holloway (shane_holloway) Date: 2005-04-13 17:21 Message: Logged In: YES user_id=283742 I would prefer to keep str's interface lean. I think binascii would be a nice home for strxor though. ---------------------------------------------------------------------- Comment By: Antti Louko (alouko) Date: 2005-04-13 14:01 Message: Logged In: YES user_id=116402 This seems good. I would also need strxor. Should we add it to the string type? ^ operator is not currently defined for strings. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1182394&group_id=5470 From noreply at sourceforge.net Fri Dec 22 21:15:36 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 12:15:36 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 20:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 20:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 22:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Fri Dec 22 22:16:30 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 13:16:30 -0800 Subject: [Patches] [ python-Patches-1097797 ] Encoding for Code Page 273 used by EBCDIC Germany Austria Message-ID: Patches item #1097797, was opened at 2005-01-07 11:44 Message generated for change (Comment added) made by mbieren You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1097797&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Michael Bierenfeld (mbieren) Assigned to: Nobody/Anonymous (nobody) Summary: Encoding for Code Page 273 used by EBCDIC Germany Austria Initial Comment: CP273.TXT to be used with gencodec.py. Enables Communication with "old" IBM Mainframes ---------------------------------------------------------------------- >Comment By: Michael Bierenfeld (mbieren) Date: 2006-12-22 21:16 Message: Logged In: YES user_id=1074928 Originator: YES Yes the patch can be included. The file should be used by gencodec.py ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2006-12-22 19:31 Message: Logged In: YES user_id=38388 Originator: NO I guess so. Note that it's not a patch, but the template to be used with gencodec.py. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 19:26 Message: Logged In: YES user_id=11375 Originator: NO Given that the patch author has provided further details, can this patch be accepted now? ---------------------------------------------------------------------- Comment By: Michael Bierenfeld (mbieren) Date: 2005-01-07 14:45 Message: Logged In: YES user_id=1074928 The encoding is used on IBM Mainframes here in Germany and Austria. Mostly on "old" VM/VSE Machines. Rawdata is transferred via MQSeries and then encoded from "cp273" to "latin_1". The file has been translated from the original IBM RFC by myself. www.unicode.org has no equivalent file on their FTP-Servers (ftp://ftp.unicode.org/Public/MAPPINGS/). CP273 is an alias for this charset. source: IBM NLS RM Vol2 SE09-8002-01, March 1990 Regards Michael ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2005-01-07 12:25 Message: Logged In: YES user_id=38388 Can you provide a reference of where and how this encoding is used ? We also need information of where the attached file originated. Thanks. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1097797&group_id=5470 From noreply at sourceforge.net Fri Dec 22 22:23:57 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 13:23:57 -0800 Subject: [Patches] [ python-Patches-738094 ] for i in range(N) optimization Message-ID: Patches item #738094, was opened at 2003-05-15 03:14 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=738094&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.3 >Status: Closed >Resolution: Rejected Priority: 2 Private: No Submitted By: Sebastien Keim (s_keim) Assigned to: Guido van Rossum (gvanrossum) Summary: for i in range(N) optimization Initial Comment: This patch is intended to special case the built-in range function in the common "for i in range(...):" construct. The goal is to make range() return an iterator instead of creating a real list, and then being able to depreciate the xrange type. It has once been suggested to make the compiler aware of the "for i in range(N):" construct and to make it able to produce optimized bytecode. But this solution is really hard to achieve because you have to ensure that the range built-in is not overridden. The patch take an opposite approach: it let the range built-in function looks at its execution context, and return an iterator if the next frame opcode to be executed is the GET_ITER opcode. Speed increase for the piece of code "for i in range(N): pass" : N (speed gain) 10 (+ 64%) 100 (+ 29%) 1000 (+ 23%) 10000 (+ 68%) 100000 (+108%) Since the patch only affect a small construct of the language, performance improvements for real applications are less impressive but they are still interesting: pystone.py (+7%) test_userstring.py (+8%) test_datetime.py (+20%) Note that the performance loss for "A = range(10)" is not measurable (less than 1%). If the patch is accepted, the same recipe may be applicable in some few other places. So the Py_IsIterationContext function must probably live somewhere else (is there a standard location for byte-code dependent stuff?). Maybe other opcodes (for sample JUMP_IF_FALSE) could provide other useful specialization contexts. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 16:23 Message: Logged In: YES user_id=6380 Originator: NO Yes, in the light of 3.0 this is unnecessary. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 13:46 Message: Logged In: YES user_id=11375 Originator: NO Should this patch simply be closed? Python 3.0 aims at fixing the range/xrange divide, doesn't it? This makes optimizations for 2.x not likely to be of great interest. ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-03-03 09:48 Message: Logged In: YES user_id=1188172 I don't know if I fully understand, but doesn't it suffice to just use xrange()? ---------------------------------------------------------------------- Comment By: Armin Rigo (arigo) Date: 2004-01-11 06:07 Message: Logged In: YES user_id=4771 Here is a safer patch. It adds a keyword argument 'iter' to range(), e.g.: >>> range(10, iter=True) and using an appropriate METH_XXX flag, the CALL_FUNCTION opcode now inserts a 'iter=True' keyword to the call when it is followed by GET_ITER. The patch doesn't live up to its performance promizes. I don't get any improvement at all on any real application. The only example it accelerates is a set of three nested loops :-( I still attach it for reference, and if someone else want to play with it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2003-07-09 12:22 Message: Logged In: YES user_id=6380 In the sake of stability for Python 2.3's accelerated release schedule, I'm postponing this until after 2.3. I'm also skeptical that it ca be absolutely correct. What if there is Python code of the form for i in some_function(): ... where some_function() is a C extension that at some point invokes range(), directly from C. Then when range() peeks in the opcode stream, it would believe that it was being called in the place of some_function(). So maybe I should just reject it as unsafe? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2003-05-18 17:18 Message: Logged In: YES user_id=6380 I'm interested, but have to ponder more, which will have to wait until I'm back from vacation. I expect that any hope to deprecate xrange() will prove naive -- people will want to pass ranges around between functions or reuse them (e.g. this happens a lot in timing tests). Maybe in Python 3.0 I can make range() act as an iterator generator. You'd have to say list(range(N)) to get an actual list then. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-17 19:25 Message: Logged In: YES user_id=80475 Assigning to Guido to see whether he is interested because it makes xrange less necessary or whether he thinks it is a horrendous hack --or maybe both ;-) ---------------------------------------------------------------------- Comment By: Sebastien Keim (s_keim) Date: 2003-05-15 11:14 Message: Logged In: YES user_id=498191 I have also thought about slicing, map and filter which could all be replaced by itertools equivalents , but I have failed to find a way to ensure that the argument lists aren't mutated during the for loop. Maybe it could be interesting to investigate into copy on write semantic for lists objects? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-05-15 10:33 Message: Logged In: YES user_id=80475 zip() would benefit greatly from your approach. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=738094&group_id=5470 From noreply at sourceforge.net Fri Dec 22 22:41:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 13:41:25 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 15:53 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 16:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 15:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 15:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 17:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 13:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 04:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 03:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 20:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 12:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 20:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Fri Dec 22 23:51:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 14:51:22 -0800 Subject: [Patches] [ python-Patches-1094542 ] add Bunch type to collections module Message-ID: Patches item #1094542, was opened at 2005-01-02 11:26 Message generated for change (Comment added) made by bediviere You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1094542&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Modules Group: Python 2.5 Status: Closed Resolution: Postponed Priority: 5 Private: No Submitted By: Steven Bethard (bediviere) Assigned to: Nobody/Anonymous (nobody) Summary: add Bunch type to collections module Initial Comment: This patch adds a proposed Bunch type to the collections module which complements Python's dict type, which provides a name-value mapping through the __getitem__ protocol, with the Bunch type, which provides an attribute-value mapping through dotted-attribute access. Example: >>> x = Bunch(spam=Bunch(rabbit=1, badger=[2, 3, 4]), ham='neewom') >>> x.spam.badger [2, 3, 4] >>> x.ham 'neewom' ---------------------------------------------------------------------- >Comment By: Steven Bethard (bediviere) Date: 2006-12-22 15:51 Message: Logged In: YES user_id=945502 Originator: YES Yeah, the PEP was never submitted because it didn't look like there was quite enough interest. Thanks for closing it. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2006-12-22 12:30 Message: Logged In: YES user_id=11375 Originator: NO It looks like the PEP was never added to the PEP index. Do you want to resubmit it? I'll close this patch per the original submitter's request. ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-02-16 00:16 Message: Logged In: YES user_id=945502 For anyone still interested, development on this type has moved to: http://namespace.python-hosting.com/ The type has been renamed Namespace. I guess this patch should be withdrawn. We'll submit a better version (with an updated PEP) when more details have been hashed out, and we have a little more use data. ---------------------------------------------------------------------- Comment By: Alan Green (alanvgreen) Date: 2005-01-26 17:50 Message: Logged In: YES user_id=1174944 AttrMap and AttributeMapping aren't great names either, because a Bunch isn't like a Map. Here's two more suggestions: - Plain or PlainObject (implying that these are plain objects that the program can adorn in any way it likes) - Holder, AttributeHolder, DataHolder, or Data (If I were in a particularly whimsical mood, I might also suggest 'Eric', after the Monty Python song "Eric the half-a-bee", because a Bunch is only half-an-object - it only has data and no functions. But I'm not, so I won't.) ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-01-25 14:01 Message: Logged In: YES user_id=945502 Yes, I definitely agree. I only used Bunch because it was the least controversial (and was the name used in the Cookbook). Unfortunately, it's also too vague. I feel a little uneasy about Record or Struct because, to me, they seem to imply an ordering of the attributes, while the attributes of the type here are unordered. Struct also already has a meaning in the stdlib which is quite different. Namespace has the right meaning, but I think it's so overloaded for XML that this could be confusing. Perhaps something like AttributeMapping? AttrMap, maybe? ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2005-01-25 13:42 Message: Logged In: YES user_id=80475 This needs a better name than Bunch. The clearly suggested a collection but is otherwise amorphous and fails to suggest any relevant properties. In language design, naming is vital. ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-01-25 13:06 Message: Logged In: YES user_id=945502 Ok, I'll post the current PEP draft below. Of course, I welcome any suggestions to make its acceptance more likely. I'll definitely update the __eq__ docstring as suggested, but I'll probably wait until I've heard what other changes should be made. ---------------------------------------------------------------------- PEP: XXX Title: Generic Object Data Type Version: $Revision: 1.0 $ Last-Modified: $Date: 2004/11/29 16:00:00 $ Author: Steven Bethard Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 29-Nov-2004 Python-Version: 2.5 Post-History: 29-Nov-2004 Abstract ======== This PEP proposes a standard library addition to support the simple creation of 'generic' objects which can be given named attributes without the need to declare a class. Such attribute-value mappings are intended to complement the name-value mappings provided by Python's builtin dict objects. Motivation ========== Python's dict objects provide a simple way of creating anonymous name-value mappings. These mappings use the __getitem__ protocol to access the value associated with a name, so that code generally appears like:: mapping['name'] Occasionally, a programmer may decide that dotted-attribute style access is more appropriate to the domain than __getitem__ style access, and that their mapping should be accessed like:: mapping.name Currently, if a Python programmer makes this design decision, they are forced to declare a new class, and then build instances of this class. When no methods are to be associated with the attribute-value mappings, declaring a new class can be overkill. This PEP proposes adding a simple type to the collections module of the standard library that can be used to build such attribute-value mappings. Providing such a type allows the Python programmer to determine which type of mapping is most appropriate to their domain and apply this choice with minimal effort. Some of the suggested uses include: Returning Named Results ----------------------- It is often appropriate for a function that returns multiple items to give names to the different items returned. The type suggested in this PEP provides a simple means of doing this that allows the returned values to be accessed in the usual attribute-style access:: >>> def f(x): ... return Bunch(double=2*x, squared=x**2) ... >>> y = f(10) >>> y.double 20 >>> y.squared 100 Representing Hierarchical Data ------------------------------ The type suggested in this PEP also allows a simple means of representing hierarchical data that allows attribute-style access:: >>> x = Bunch(spam=Bunch(rabbit=1, badger=[2, 3, 4]), ham='neewom') >>> x.spam.badger [2, 3, 4] >>> x.ham 'neewom' Rationale ========= As Bunch objects are intended primarily to replace simple, data-only classes, simple Bunch construction was a primary concern. As such, the Bunch constructor supports creation from keyword arguments, dicts, and sequences of (attribute, value) pairs:: >>> Bunch(eggs=1, spam=2, ham=3) Bunch(eggs=1, ham=3, spam=2) >>> Bunch({'eggs':1, 'spam':2, 'ham':3}) Bunch(eggs=1, ham=3, spam=2) >>> Bunch([('eggs',1), ('spam',2), ('ham',3)]) Bunch(eggs=1, ham=3, spam=2) To allow attribute-value mappings to be easily combined, the update method of Bunch objects supports similar arguments. If Bunch objects are used to represent hierarchical data, comparison of such objects becomes a concern. For this reason, Bunch objects support object equality:: >>> x = Bunch(parrot=Bunch(lumberjack=True, spam=42), peng='shrub') >>> y = Bunch(peng='shrub', parrot=Bunch(spam=42, lumberjack=True)) >>> z = Bunch(parrot=Bunch(lumberjack=True), peng='shrub') >>> x == y True >>> x == z False Note that support for the various mapping methods, e.g. __(get|set|del)item__, __len__, __iter__, __contains__, items, keys, values, etc. was intentionally omitted as these methods did not seem to be necessary for the core uses of an attribute-value mapping. If such methods are truly necessary for a given use case, this may suggest that a dict object is a more appropriate type for that use. Examples ========= Converting an XML DOM tree into a tree of nested Bunch objects:: >>> import xml.dom.minidom >>> def getbunch(element): ... result = Bunch() ... if element.attributes: ... result.update(element.attributes.items()) ... children = {} ... for child in element.childNodes: ... if child.nodeType == xml.dom.minidom.Node.TEXT_NODE: ... children.setdefault('text', []).append( ... child.nodeValue) ... else: ... children.setdefault(child.nodeName, []).append( ... getbunch(child)) ... result.update(children) ... return result ... >>> doc = xml.dom.minidom.parseString("""\ ... ... ... a text 1 ... ... b text ... a text 2 ... ... c text ... """) >>> b = getbunch(doc.documentElement) >>> b.a[0].b[1] Bunch(attr_b=u'3', text=[u' b text ']) Reference Implementation ======================== The code is available as SourceForge patch 1094542 [1]_. Open Issues =========== What should the type be named? Some suggestions include 'Bunch', 'Record', 'Struct' and 'Namespace'. Where should the type be placed? The current suggestion is the collections module. References ========== .. [1] http://sourceforge.net/tracker/index.php?func=detail&aid=1094542&group_id=5470&atid=305470 .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: ---------------------------------------------------------------------- Comment By: Alan Green (alanvgreen) Date: 2005-01-24 05:34 Message: Logged In: YES user_id=1174944 Some thoughts while we wait for the PEP: 1. This doesn't look like a collection class to me. It's more of a convenient substitute for a first-class object. The PEP would need to include a rationale as to why this class is in the collections module. 2. It may be desireable to make the core parts of Bunch (equality test, repr, and update) available as functions that other classes can use if appropriate. This might help developers building objects more complex than a Bunch. Then again, maybe not, but I'd like to see the PEP address this. 3. The docstring on __eq__ should explicitly say what consitutes equality for bunches: both bunches have the same attributes and the same values for those attributes. 4. It's easy enough to convert a dict to a Bunch (via the Bunch constructor), but I would have expected that there be a way to convert a Bunch to a dict. Overall, a useful concept, but I'd like to read the PEP - you could always upload your draft to this patch item :) ---------------------------------------------------------------------- Comment By: Steven Bethard (bediviere) Date: 2005-01-10 08:15 Message: Logged In: YES user_id=945502 I submitted a PEP for it on 2 Jan 2005, but I haven't heard back from peps at python.org yet. Sorry, I didn't realize it might take so long to get a PEP number. ---------------------------------------------------------------------- Comment By: Michael Chermside (mcherm) Date: 2005-01-10 07:08 Message: Logged In: YES user_id=99874 Would someone be willing to provide the motivation for adding this class? I'm certainly willing to listen, but I'm not convinced this is worth adding to the std library. (I guess that's a -0 vote.) ---------------------------------------------------------------------- Comment By: Georg Brandl (birkenfeld) Date: 2005-01-03 10:38 Message: Logged In: YES user_id=1188172 Let me add that the main functionality consists in the easy initializing and updating (otherwise, you just could use an empty class). I'm +1 on the class, but I would call it `bunch'. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1094542&group_id=5470 From noreply at sourceforge.net Sat Dec 23 04:33:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 22 Dec 2006 19:33:25 -0800 Subject: [Patches] [ python-Patches-1616125 ] Cached globals+builtins lookup optimization Message-ID: Patches item #1616125, was opened at 2006-12-14 16:44 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrea Griffini (ag6502) Assigned to: Nobody/Anonymous (nobody) Summary: Cached globals+builtins lookup optimization Initial Comment: This patch implements a speed optimization by introducing a timestamp on dictionaries and by caching lookups of constants so that lookup doesn't need to be repeated if the dictionary didn't change. Currently the patch implements the cached lookup only for the LOAD_GLOBAL opcode and stores the cache as two extra member in the code object. I'm going to investigate if LOAD_ATTR in the special case of a module is worth caching too. A cache entry for LOAD_GLOBAL is defined by two timestamps (Py_ssize_t) and a *borrowed* pointer to a PyObject. The use of a borrowed pointer is safe because the value is used only if the dictionaries didn't change, so the result of lookup must still be alive because the reference in the dictionary that returned the cached value it is still in place. The assumptions are: - a LOAD_GLOBAL opcode always looks for the same symbol (i.e. co_names never changes) - the dictionary being searched (globals and builtins) are mostly constant On my system I got a speedup of 8%-10% on a real program (a tcp/ip server that handles an in-memory database of python objects) on the checkpoint load + rollforward steps (a few millions LOAD_GLOBAL operations in total) and the varying percentage depends on which compiler options are used to build python. On another little test program that calls a function def f(x): return math.sin(x) + math.cos(x) + abs(x) one million times the speedup with the default makefile is about 7%. To enable the patch the makefile should define the CACHED_LOOKUPS symbol. The change to dictobject also exports the timestamp, at the python level; other changes are instead invisible at the python level. I'm not sure at all exporting dict.timestamp() is a good idea however... Note that after applying the patch you may find an error in userdict if you simply use "make" (there's probably some dependency problem somewhere). After make clean + make the test suite runs fine on my system. I'm interested in hearing if also other platforms do actually see a real performance improvement. Andrea ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-22 19:33 Message: Logged In: YES user_id=33168 Originator: NO I only reviewed the code in the patch. I have not thought conceptually about this, whether it's a good idea, how to break it, etc. I also did not look to verify all the necessary dict methods were updated to add TOUCHED. Have you profiled the slow down to dicts in normal cases? Things like: d = {} d[0] = 0 etc? All dict operations are going to be a tiny bit slower due to an additional assignment. It's probably not measurable, but you should still verify this. What is LOAD_FAST_HACK in ceval.c? It doesn't seem necessary for this patch. Please revert any whitespace/formatting changes that aren't part of the patch (there are a couple in dictobject.c and codeobject.c). It seems like a little structure would be better than parallel arrays. This would reduce some calculations and reduce the number of mallocs. Also there would only be a single assignment in the eval loop and the names would probably wind up being more clear. Would it be better to use PyObject_Malloc rather than PyMem_Malloc? It should be faster for sizes <= 256. (Despite the name, it doesn't have to handle PyObjects.) Why not use a size_t instead of Py_ssize_t for the timestamp? This is conceptually unsigned, no? Use the macro version PyTuple_GET_SIZE(names); in codeobject.c. It will be a little faster. If the cache allocs fail when allocating a code object, ceval will crash due to not checking for NULL. As for the timestamp, there should probably be a macro to access the field for a dict. This macro should be used in ceval.c/codeobject.c for getting/setting it. When returning the timestamp in dictobject.c you shouldn't cast it to a long. Use the appropriate method such as PyInt_FromSsize_t. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 14:57 Message: Logged In: YES user_id=21627 Originator: NO I think restricting yourself to 32-bit counters is fine (it's also an issue of memory overhead as you have one of these per access to a global, and per dictionary); you have to deal with the overflow, though. I suggest to use an unsigned value, e.g. size_t. Overflowing should be dealt with through visiting all objects, as you say. It could either happen right when it overflows, or it could get integrated into GC, and only reset when the GC runs. You'd then need to account for those periods when it already has overflowed, but GC hasn't run yet; in these periods, counting should be "on hold". ---------------------------------------------------------------------- Comment By: Andrea Griffini (ag6502) Date: 2006-12-16 09:13 Message: Logged In: YES user_id=1668946 Originator: YES I am experimenting with long long timestamps (based on "unsigned long long" if available or explicitly on two "unsigned long" otherwise). An alternative to slowing down lookups by using 64 bits on 32-bit computers is just using a single 32-bit counter and trigger a "cache flush" when the timestamp rolls over by visiting all live dictionaries and code objects. This would be IMO a *very* infrequent operation (for example on my P4/2.4GHz just doing nothing else but dict updates getting to 2**32 updates would require 14 minutes). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 03:01 Message: Logged In: YES user_id=21627 Originator: NO How does that deal with timestamp wraparound? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 From noreply at sourceforge.net Sat Dec 23 14:16:26 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 05:16:26 -0800 Subject: [Patches] [ python-Patches-1621265 ] Auto-completion list placement Message-ID: Patches item #1621265, was opened at 2006-12-23 15:16 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621265&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Nobody/Anonymous (nobody) Summary: Auto-completion list placement Initial Comment: The completion list placement logic is currently very primitive. Sometimes the completion list extends outside of the editor window, or even outside the screen so that part of it is not visible. This patch places the completion list inside the text widget if possible, above or below the typed text, as appropriate. It will not hide the currently typed text in any case. End-cases: * If the text widget is not high enough, the completion list is placed below the currently typed text. * If the text widget is not wide enough, the completion list is aligned with the left-most edge of the text widget. In any case the size of the completion list is not altered to fit the text widget, though this would probably be the best solution when the text widget is too small. This means that with this patch, the completion list may extend beyond the text widget if it is small. I have not implemented resizing of the list this since the list's size is currently not set at all - Tk's default is used. Changing it anywhere in the code would require setting it elsewhere, hard-coding a default or making it configurable, etc. Comments please! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621265&group_id=5470 From noreply at sourceforge.net Sat Dec 23 14:17:36 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 05:17:36 -0800 Subject: [Patches] [ python-Patches-1621265 ] Auto-completion list placement Message-ID: Patches item #1621265, was opened at 2006-12-23 15:16 Message generated for change (Settings changed) made by taleinat You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621265&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: IDLE Group: Python 2.6 Status: Open Resolution: None >Priority: 2 Private: No Submitted By: Tal Einat (taleinat) Assigned to: Nobody/Anonymous (nobody) Summary: Auto-completion list placement Initial Comment: The completion list placement logic is currently very primitive. Sometimes the completion list extends outside of the editor window, or even outside the screen so that part of it is not visible. This patch places the completion list inside the text widget if possible, above or below the typed text, as appropriate. It will not hide the currently typed text in any case. End-cases: * If the text widget is not high enough, the completion list is placed below the currently typed text. * If the text widget is not wide enough, the completion list is aligned with the left-most edge of the text widget. In any case the size of the completion list is not altered to fit the text widget, though this would probably be the best solution when the text widget is too small. This means that with this patch, the completion list may extend beyond the text widget if it is small. I have not implemented resizing of the list this since the list's size is currently not set at all - Tk's default is used. Changing it anywhere in the code would require setting it elsewhere, hard-coding a default or making it configurable, etc. Comments please! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621265&group_id=5470 From noreply at sourceforge.net Sat Dec 23 16:53:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 07:53:07 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 15:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 21:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 20:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 20:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 22:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Sat Dec 23 17:52:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 08:52:25 -0800 Subject: [Patches] [ python-Patches-1230446 ] tarfile.py: ExFileObject\'s tell() is wrong after readline() Message-ID: Patches item #1230446, was opened at 2005-06-30 18:23 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1230446&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile.py: ExFileObject\'s tell() is wrong after readline() Initial Comment: This patch is intended to be the solution to a problem that Niklas Volbers reported on comp.lang.python on June 1st. ExFileObject is the pseudo file object that is returned by TarFile.extractfile() and allows a user to read a file from the tar archive using a file interface. Niklas discovered that the tell() method didn't give correct results if he used it in conjunction with readline(). The cause for this is that readline() buffers the file data in 100 char blocks to be able to split it into lines. Thus, tell() always returns the file position in 100 byte steps. While I was looking for a fix to that problem in tarfile.py, I discovered that ExFileObject has another flaw as well: I read in readline()'s docstring that mixing calls to read() and readline() is not allowed, which is nowhere documented. I decided to put a little effort into a rewrite of ExFileObject. Here it is, tests included. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-23 17:52 Message: Logged In: YES user_id=642936 Originator: YES Committed as rev. 53153 and rev. 53154 (2.5). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1230446&group_id=5470 From noreply at sourceforge.net Sat Dec 23 19:14:52 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 10:14:52 -0800 Subject: [Patches] [ python-Patches-1262036 ] tarfile: fix for bug #1257255 Message-ID: Patches item #1262036, was opened at 2005-08-17 14:33 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1262036&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Lars Gust?bel (gustaebel) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile: fix for bug #1257255 Initial Comment: (See bug #1257255 for a detailed description of the problem.) While the problem of the OP is that tarfile won't work when the current working dir has been removed from the filesystem, the more important one for me is that the test in the add() method, that should prevent the archive from being added to itself, will only succeed by accident. So, I decided to save the TarFile's name as an absolute path from the beginning to ensure that the archive cannot be added to itself even if the cwd changed during the operation. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-23 19:14 Message: Logged In: YES user_id=642936 Originator: YES Committed a fix based on tarfile-name-2.diff to rev. 53155 and rev. 53156 (2.5). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-08-24 08:49 Message: Logged In: YES user_id=21627 The patch is actually incorrect. You cannot tar broken symlinks anymore, as os.path.samefile will raise an OSError (no such file or directory). Also, os.path.samefile doesn't exist on Windows. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2005-08-24 08:08 Message: Logged In: YES user_id=21627 Thanks for the patch. Committed as tarfile.py 1.30 NEWS 1.1339 tarfile.py 1.21.2.3 NEWS 1.1193.2.75 ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1262036&group_id=5470 From noreply at sourceforge.net Sat Dec 23 20:03:05 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 11:03:05 -0800 Subject: [Patches] [ python-Patches-1587674 ] Patch for #1586414 to avoid fragmentation on Windows Message-ID: Patches item #1587674, was opened at 2006-10-31 06:05 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1587674&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Enoch Julias (enochjul) >Assigned to: Lars Gust?bel (gustaebel) Summary: Patch for #1586414 to avoid fragmentation on Windows Initial Comment: Add a call to file.truncate() to inform Windows of the size of the target file in makefile(). This helps guide cluster allocation in NTFS to avoid fragmentation. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-23 20:03 Message: Logged In: YES user_id=642936 Originator: NO Any progress on this one? ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-11-08 22:30 Message: Logged In: YES user_id=642936 You both still fail to convince me and I still don't see need for action. The only case ATM where this addition makes sense (in your opinion) is the Windows OS when using the NTFS filesystem and certain conditions are met. NTFS has a preallocation algorithm to deal with this. We don't know if there is any advantage on FAT filesystems. On Linux for example there is a plethora of supported filesystems. Some of them may take advantage, others may not. Who knows? We can't even detect which filesystem type we are currently writing to. Apart from that, the behaviour of truncate(arg) with arg > filesize seems to be system-dependent. So, IMO this is a very special optimization targeted at a single platform. The TarFile class is easily subclassable, just override the makefile() method and add the two lines of code. I think that's what ActiveState's Python Cookbook is for. BTW, I like my files to grow bit by bit. In case of an error, I can detect if a file was not extracted completely by comparing the file sizes. Furthermore, a file that grows is more common and more what a programmer who uses this module might expect. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2006-11-08 17:33 Message: Logged In: YES user_id=341410 I disagree with user gustaebel. We should be adding automatic truncate calls for all possible supported platforms, in all places where it could make sense. Be it in tarfile, zipfile, where ever we can. It would make sense to write a function that can be called by all of those modules so that there is only one place to update if/when changes occur. If the function were not part of the public Python API, then it wouldn't need to wait until 2.6, unless it were considered a feature addition rather than bugfix. One would have to wait on a response from Martin or Anthony to know which it was, though I couldn't say for sure if operations that are generally performance enhancing are bugfixes or feature additions. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-11-06 22:57 Message: Logged In: YES user_id=642936 Personally, I think disk defragmenters are evil ;-) They create the need that they are supposed to satisfy at the same time. On Linux we have no defragmenters, so we don't bother about it. I think your proposal is some kind of a performance hack for a particular filesystem. In principle, this problem exists for all filesystems on all platforms. Fragmentation is IMO a filesystem's problem and is not so much a state but more like a process. Filesystem fragment over time and you can't do anything about it. For those people who care, disk fragmenter were invented. It is not tarfile.py's job to care about a fragmented filesystem, that's simply too low level. I admit that it is a small patch, but I'm -1 on having this applied. ---------------------------------------------------------------------- Comment By: Enoch Julias (enochjul) Date: 2006-11-06 18:19 Message: Logged In: YES user_id=6071 I have not really tested FAT/FAT32 yet as I don't use these filesystems now. The Disk Defragmenter tool in Windows 2000/XP shows the number of files/directories fragmented in its report. NTFS does handle growing files, but the operating system can only do so much without knowing the size of the file. Extracting from archives consisting of only several files does not cause fragmentation. However, if the archive has many files, it is much more likely that the default algorithm will fail to allocate contiguous clusters for some files. It may also depend on the amount of free space fragmentation on a particular partition and whether other processes are writing to other files in the same partition. Some details of the cluster allocation algorithm used in Windows can be found at http://support.microsoft.com/kb/841551. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-11-01 16:27 Message: Logged In: YES user_id=642936 Is this merely an NTFS problem or is it the same with FAT fs? How do you detect file fragmentation? Doesn't this problem apply to all other modules or scripts that write to file objects as well? Shouldn't a decent filesystem be able to handle growing files in a correct manner? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1587674&group_id=5470 From noreply at sourceforge.net Sat Dec 23 20:05:39 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 11:05:39 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 19:05 Message: Logged In: YES user_id=24100 Originator: YES Initial patch to implement keyword-only arguments and annotations support for pydoc and inspect. Tests do not exercise these features, yet. Output for annotations that are types is special cased so that for: def intmin(*a: int) -> int: pass ...help(intmin) will display: intmin(*a: int) -> int File Added: pydoc.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 15:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 21:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 20:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 20:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 22:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Sat Dec 23 21:01:44 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 12:01:44 -0800 Subject: [Patches] [ python-Patches-1621421 ] normalize namespace from minidom Message-ID: Patches item #1621421, was opened at 2006-12-23 20:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Pacheco (paulpach) Assigned to: Nobody/Anonymous (nobody) Summary: normalize namespace from minidom Initial Comment: This is a patch that fixes bug 1371937, along with a test case that reproduces the original problem. The patch is a port of the algorithm described here pretty much verbatim: http://www.w3.org/TR/DOM-Level-3-Core/namespaces-algorithms.html#normalizeDocumentAlgo to resolve namespace declarations. The patch also calls the normalizeNamespace method before calling writexml Note the patch is against minidom that comes with python 2.5, NOT WITH PYXML. I need this for an embedded project where pyxml is too big. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 From noreply at sourceforge.net Sat Dec 23 21:02:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 12:02:43 -0800 Subject: [Patches] [ python-Patches-1621421 ] normalize namespace from minidom Message-ID: Patches item #1621421, was opened at 2006-12-23 20:01 Message generated for change (Comment added) made by paulpach You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Pacheco (paulpach) Assigned to: Nobody/Anonymous (nobody) Summary: normalize namespace from minidom Initial Comment: This is a patch that fixes bug 1371937, along with a test case that reproduces the original problem. The patch is a port of the algorithm described here pretty much verbatim: http://www.w3.org/TR/DOM-Level-3-Core/namespaces-algorithms.html#normalizeDocumentAlgo to resolve namespace declarations. The patch also calls the normalizeNamespace method before calling writexml Note the patch is against minidom that comes with python 2.5, NOT WITH PYXML. I need this for an embedded project where pyxml is too big. ---------------------------------------------------------------------- >Comment By: Paul Pacheco (paulpach) Date: 2006-12-23 20:02 Message: Logged In: YES user_id=794762 Originator: YES File Added: normalizeNamespace.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 From noreply at sourceforge.net Sat Dec 23 23:52:35 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 23 Dec 2006 14:52:35 -0800 Subject: [Patches] [ python-Patches-1616125 ] Cached globals+builtins lookup optimization Message-ID: Patches item #1616125, was opened at 2006-12-15 01:44 Message generated for change (Comment added) made by ag6502 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrea Griffini (ag6502) Assigned to: Nobody/Anonymous (nobody) Summary: Cached globals+builtins lookup optimization Initial Comment: This patch implements a speed optimization by introducing a timestamp on dictionaries and by caching lookups of constants so that lookup doesn't need to be repeated if the dictionary didn't change. Currently the patch implements the cached lookup only for the LOAD_GLOBAL opcode and stores the cache as two extra member in the code object. I'm going to investigate if LOAD_ATTR in the special case of a module is worth caching too. A cache entry for LOAD_GLOBAL is defined by two timestamps (Py_ssize_t) and a *borrowed* pointer to a PyObject. The use of a borrowed pointer is safe because the value is used only if the dictionaries didn't change, so the result of lookup must still be alive because the reference in the dictionary that returned the cached value it is still in place. The assumptions are: - a LOAD_GLOBAL opcode always looks for the same symbol (i.e. co_names never changes) - the dictionary being searched (globals and builtins) are mostly constant On my system I got a speedup of 8%-10% on a real program (a tcp/ip server that handles an in-memory database of python objects) on the checkpoint load + rollforward steps (a few millions LOAD_GLOBAL operations in total) and the varying percentage depends on which compiler options are used to build python. On another little test program that calls a function def f(x): return math.sin(x) + math.cos(x) + abs(x) one million times the speedup with the default makefile is about 7%. To enable the patch the makefile should define the CACHED_LOOKUPS symbol. The change to dictobject also exports the timestamp, at the python level; other changes are instead invisible at the python level. I'm not sure at all exporting dict.timestamp() is a good idea however... Note that after applying the patch you may find an error in userdict if you simply use "make" (there's probably some dependency problem somewhere). After make clean + make the test suite runs fine on my system. I'm interested in hearing if also other platforms do actually see a real performance improvement. Andrea ---------------------------------------------------------------------- >Comment By: Andrea Griffini (ag6502) Date: 2006-12-23 23:52 Message: Logged In: YES user_id=1668946 Originator: YES I was already changing the code in the direction you're pointing out. I attached the current version (still not complete!) of the patch; more specifically: - Cached lookups are now stored in an array of structures instead than in two separate arrays - Macros are used for handling of cache entries - The timestamp is now unsigned and there is optional support for 64-bit timestamps - Support for stalling timestamping at 32 bit wrap point - Removed the LOAD_FAST_HACK (was just an experiment with oprofile; was left there by mistake) - Fixed memory allocation failures handling in code object creation - Removed exporting of timestamp value at the python level - Used as cache size the largest LOAD_ATTR/LOAD_GLOBAL entry found Still missing are the sorting of co_names to pack cache slots and the cache reset at gc time if the dictionary timestamp is stalled. The cached lookup can be used at two levels: -DCACHED_LOOKUPS enables caching of LOAD_GLOBAL results -DCACHED_MODULE_LOOKUPS (in addition to -DCACHED_LOOKUPS) also caches the result of a lookup in a module. Speed results are sometimes very interesting and sometimes strange; I found measurable differences just moving around statements between logically equivalent places after checking what gcc was doing with register allocation (probably those places were indeed not equivalent if taking in account aliasing that isn't going to happen but that a c compiler must assume as possible). I have no idea if the speedups I've measured are better or worse on other processors/architectures. File Added: cached_lookups_10.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-23 04:33 Message: Logged In: YES user_id=33168 Originator: NO I only reviewed the code in the patch. I have not thought conceptually about this, whether it's a good idea, how to break it, etc. I also did not look to verify all the necessary dict methods were updated to add TOUCHED. Have you profiled the slow down to dicts in normal cases? Things like: d = {} d[0] = 0 etc? All dict operations are going to be a tiny bit slower due to an additional assignment. It's probably not measurable, but you should still verify this. What is LOAD_FAST_HACK in ceval.c? It doesn't seem necessary for this patch. Please revert any whitespace/formatting changes that aren't part of the patch (there are a couple in dictobject.c and codeobject.c). It seems like a little structure would be better than parallel arrays. This would reduce some calculations and reduce the number of mallocs. Also there would only be a single assignment in the eval loop and the names would probably wind up being more clear. Would it be better to use PyObject_Malloc rather than PyMem_Malloc? It should be faster for sizes <= 256. (Despite the name, it doesn't have to handle PyObjects.) Why not use a size_t instead of Py_ssize_t for the timestamp? This is conceptually unsigned, no? Use the macro version PyTuple_GET_SIZE(names); in codeobject.c. It will be a little faster. If the cache allocs fail when allocating a code object, ceval will crash due to not checking for NULL. As for the timestamp, there should probably be a macro to access the field for a dict. This macro should be used in ceval.c/codeobject.c for getting/setting it. When returning the timestamp in dictobject.c you shouldn't cast it to a long. Use the appropriate method such as PyInt_FromSsize_t. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 23:57 Message: Logged In: YES user_id=21627 Originator: NO I think restricting yourself to 32-bit counters is fine (it's also an issue of memory overhead as you have one of these per access to a global, and per dictionary); you have to deal with the overflow, though. I suggest to use an unsigned value, e.g. size_t. Overflowing should be dealt with through visiting all objects, as you say. It could either happen right when it overflows, or it could get integrated into GC, and only reset when the GC runs. You'd then need to account for those periods when it already has overflowed, but GC hasn't run yet; in these periods, counting should be "on hold". ---------------------------------------------------------------------- Comment By: Andrea Griffini (ag6502) Date: 2006-12-16 18:13 Message: Logged In: YES user_id=1668946 Originator: YES I am experimenting with long long timestamps (based on "unsigned long long" if available or explicitly on two "unsigned long" otherwise). An alternative to slowing down lookups by using 64 bits on 32-bit computers is just using a single 32-bit counter and trigger a "cache flush" when the timestamp rolls over by visiting all live dictionaries and code objects. This would be IMO a *very* infrequent operation (for example on my P4/2.4GHz just doing nothing else but dict updates getting to 2**32 updates would require 14 minutes). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 12:01 Message: Logged In: YES user_id=21627 Originator: NO How does that deal with timestamp wraparound? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 From noreply at sourceforge.net Sun Dec 24 16:29:32 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 24 Dec 2006 07:29:32 -0800 Subject: [Patches] [ python-Patches-1621421 ] normalize namespace from minidom Message-ID: Patches item #1621421, was opened at 2006-12-23 20:01 Message generated for change (Comment added) made by paulpach You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Pacheco (paulpach) Assigned to: Nobody/Anonymous (nobody) Summary: normalize namespace from minidom Initial Comment: This is a patch that fixes bug 1371937, along with a test case that reproduces the original problem. The patch is a port of the algorithm described here pretty much verbatim: http://www.w3.org/TR/DOM-Level-3-Core/namespaces-algorithms.html#normalizeDocumentAlgo to resolve namespace declarations. The patch also calls the normalizeNamespace method before calling writexml Note the patch is against minidom that comes with python 2.5, NOT WITH PYXML. I need this for an embedded project where pyxml is too big. ---------------------------------------------------------------------- >Comment By: Paul Pacheco (paulpach) Date: 2006-12-24 15:29 Message: Logged In: YES user_id=794762 Originator: YES File Added: normalizeNamespace.diff ---------------------------------------------------------------------- Comment By: Paul Pacheco (paulpach) Date: 2006-12-23 20:02 Message: Logged In: YES user_id=794762 Originator: YES File Added: normalizeNamespace.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 From noreply at sourceforge.net Sun Dec 24 16:32:01 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 24 Dec 2006 07:32:01 -0800 Subject: [Patches] [ python-Patches-1621421 ] normalize namespace from minidom Message-ID: Patches item #1621421, was opened at 2006-12-23 20:01 Message generated for change (Comment added) made by paulpach You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: XML Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Paul Pacheco (paulpach) Assigned to: Nobody/Anonymous (nobody) Summary: normalize namespace from minidom Initial Comment: This is a patch that fixes bug 1371937, along with a test case that reproduces the original problem. The patch is a port of the algorithm described here pretty much verbatim: http://www.w3.org/TR/DOM-Level-3-Core/namespaces-algorithms.html#normalizeDocumentAlgo to resolve namespace declarations. The patch also calls the normalizeNamespace method before calling writexml Note the patch is against minidom that comes with python 2.5, NOT WITH PYXML. I need this for an embedded project where pyxml is too big. ---------------------------------------------------------------------- >Comment By: Paul Pacheco (paulpach) Date: 2006-12-24 15:32 Message: Logged In: YES user_id=794762 Originator: YES File Added: test_toxml.py ---------------------------------------------------------------------- Comment By: Paul Pacheco (paulpach) Date: 2006-12-24 15:29 Message: Logged In: YES user_id=794762 Originator: YES File Added: normalizeNamespace.diff ---------------------------------------------------------------------- Comment By: Paul Pacheco (paulpach) Date: 2006-12-23 20:02 Message: Logged In: YES user_id=794762 Originator: YES File Added: normalizeNamespace.diff ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1621421&group_id=5470 From noreply at sourceforge.net Wed Dec 27 04:34:40 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 26 Dec 2006 19:34:40 -0800 Subject: [Patches] [ python-Patches-1182394 ] HMAC hexdigest and general review Message-ID: Patches item #1182394, was opened at 2005-04-13 13:08 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1182394&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Shane Holloway (shane_holloway) Assigned to: A.M. Kuchling (akuchling) Summary: HMAC hexdigest and general review Initial Comment: This patch is primarily aimed at speeding up the HMAC.hexdigest method. The current implementation is:: return "".join([hex(ord(x))[2:].zfill(2) for x in tuple(self.digest())]) Which is correct, but a little too clever. Especially when other tools are available. The attached patch adds HMAC._current that returns the hash object combined from self.inner and self.outter to be used by HMAC.digest and HMAC.hexdigest. This results in an 11 fold speed improvement for md5 and sha digestmod's while also making the code a little easier. Other review points for the HMAC module include removing "_secret_backdoor_key" in favor of using "None" as the sentinel. In this case, it makes sense to use None because it is an invalid key, avoids a global lookup in py2.4 and beyond, and is just as readable. I also moved blocksize to a class level constant. This allows implementations to create different blocksize hashes by inheriting and changing this value. This also involved a change to copy() to use self.__class__(None) instead of directly calling HMAC(None). The final comment is that it would be nice to have a C-level implementation of _strxor in binascii for this module and possibly other uses. However, I don't really know a significantly faster way of implementing this method in python. Thanks, -Shane Holloway ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-12-26 22:34 Message: Logged In: YES user_id=11375 Originator: NO Applied the blocksize change in rev. 53159. Applied the _current change in rev 53160. Thanks for your patch! ---------------------------------------------------------------------- Comment By: Shane Holloway (shane_holloway) Date: 2005-07-26 13:20 Message: Logged In: YES user_id=283742 Yes, I was reviewing HMAC to learn how it works, and to see if I could use it for a distributed verification algorithm. I will be make a fair number of hmac.copy() calls, but it will by no means be critical. Mainly, I found the implementation more than a little clunky to read, and significantly slower than it could be. ---------------------------------------------------------------------- Comment By: A.M. Kuchling (akuchling) Date: 2005-06-16 14:08 Message: Logged In: YES user_id=11375 The HMAC._current() change and adding self.blocksize both look reasonable. I don't see the point of the _secret_backdoor_key change, though; accidentally using None or calling HMAC() will get an object that fails to work, and the speedup seems too small to compensate for this. Do you have a use case that requires making lots of HMAC.copy() calls? If the backdoor_key gets taken out, I don't think we need to keep the name around for backward compatibility; no callers of the module should have been using it. ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-04-26 12:32 Message: Logged In: YES user_id=341410 Obviously "always have a length of 4" should have been "always have a length of zero modulo 4". ---------------------------------------------------------------------- Comment By: Josiah Carlson (josiahcarlson) Date: 2005-04-26 03:10 Message: Logged In: YES user_id=341410 I think the patch looks good. Though perhaps leaving a... _secret_backdoor_key = None ... for backwards compatibility is a good idea. The fastest method I've found (in pure Python) for xoring strings is to use an array.array, with the largest typecode that is a divisor of your sequence (cooking your strings to always have a length of 4 and to use signed 4 byte longs seems to work well on PII/PIII based systems). Alternatively, writing it in C and compiling it with the new GCC 4 will offer automatic vectorization, and save allocating Python integers. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2005-04-14 03:54 Message: Logged In: YES user_id=6656 Attempting to add the ^ for strings is a non-starter. No opinion on the rest of the patch. ---------------------------------------------------------------------- Comment By: Shane Holloway (shane_holloway) Date: 2005-04-13 17:21 Message: Logged In: YES user_id=283742 I would prefer to keep str's interface lean. I think binascii would be a nice home for strxor though. ---------------------------------------------------------------------- Comment By: Antti Louko (alouko) Date: 2005-04-13 14:01 Message: Logged In: YES user_id=116402 This seems good. I would also need strxor. Should we add it to the string type? ^ operator is not currently defined for strings. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1182394&group_id=5470 From noreply at sourceforge.net Wed Dec 27 11:38:19 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 02:38:19 -0800 Subject: [Patches] [ python-Patches-1504073 ] Patch for 1496501 tarfile opener order Message-ID: Patches item #1504073, was opened at 2006-06-10 19:45 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1504073&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Accepted Priority: 5 Private: No Submitted By: Jack Diederich (jackdied) Assigned to: Lars Gust?bel (gustaebel) Summary: Patch for 1496501 tarfile opener order Initial Comment: when passing a fileobj into tarfile.open() without specifying the type open() will try openers in their dict key order from TarFile.OPEN_METH. This can fail if the order changes. This patch adds a tell() and seek() on failure of openers. fileobjs must already support these methods to work so this makes no new requirements. OpenOrderTest uses a dict-alike with shuffled keys to try and provoke the wrong orders for openers. Let me know if it looks good and I'll check it in. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-27 11:38 Message: Logged In: YES user_id=642936 Originator: NO Made the testcase simpler and checked it in as rev. 53161 and rev. 53162 (2.5). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1504073&group_id=5470 From noreply at sourceforge.net Wed Dec 27 18:52:23 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 09:52:23 -0800 Subject: [Patches] [ python-Patches-1605192 ] Make Imap Error more helpful Message-ID: Patches item #1605192, was opened at 2006-11-29 05:52 Message generated for change (Comment added) made by mark-roberts You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1605192&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Guettler (guettli) Assigned to: Nobody/Anonymous (nobody) Summary: Make Imap Error more helpful Initial Comment: Hi, In my app I got this error: imaplib.error: command SEARCH illegal in state AUTH. but I did successfully login. After reading the source of imaplib.py I realized that this was meant: imaplib.error: command SEARCH illegal in state AUTH. Allowed after: SELECTED Attached is a small patch to make the error message more helpful. ---------------------------------------------------------------------- Comment By: Mark Roberts (mark-roberts) Date: 2006-12-27 11:52 Message: Logged In: YES user_id=1591633 Originator: NO The patch looks great (Not as though I have a lot of say to say so..), but it seems incongruous that you say "allowed after" when it might seem more explicit to say "Allowed in states", like so: "Allowed in states: %s" % (", ".join(Commands[command]) Otherwise, I'm all for more explicit error messages! - Mark ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1605192&group_id=5470 From noreply at sourceforge.net Thu Dec 28 01:40:19 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 16:40:19 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 15:53 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 19:40 Message: Logged In: YES user_id=6380 Originator: NO I believe I've found a leak in the code that adds annotations to a function object. See this session: >>> x = object() >>> import sys >>> sys.getrefcount(x) 2 >>> for i in range(100): ... def f(x: x): pass ... >>> del f >>> sys.getrefcount(x) 102 >>> At first I thought this could be due to the code added to the MAKE_FUNCTION opcode, but I don't see a leak there. More likely func_annotations is not being freed when a function object is deleted. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 14:05 Message: Logged In: YES user_id=24100 Originator: YES Initial patch to implement keyword-only arguments and annotations support for pydoc and inspect. Tests do not exercise these features, yet. Output for annotations that are types is special cased so that for: def intmin(*a: int) -> int: pass ...help(intmin) will display: intmin(*a: int) -> int File Added: pydoc.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 10:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 16:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 15:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 15:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 17:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 13:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 04:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 03:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 20:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 12:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 20:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 28 02:28:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 17:28:53 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-28 01:28 Message: Logged In: YES user_id=24100 Originator: YES Fixed in latest patch. Also added VISIT call for func_annotations. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-28 00:40 Message: Logged In: YES user_id=6380 Originator: NO I believe I've found a leak in the code that adds annotations to a function object. See this session: >>> x = object() >>> import sys >>> sys.getrefcount(x) 2 >>> for i in range(100): ... def f(x: x): pass ... >>> del f >>> sys.getrefcount(x) 102 >>> At first I thought this could be due to the code added to the MAKE_FUNCTION opcode, but I don't see a leak there. More likely func_annotations is not being freed when a function object is deleted. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 19:05 Message: Logged In: YES user_id=24100 Originator: YES Initial patch to implement keyword-only arguments and annotations support for pydoc and inspect. Tests do not exercise these features, yet. Output for annotations that are types is special cased so that for: def intmin(*a: int) -> int: pass ...help(intmin) will display: intmin(*a: int) -> int File Added: pydoc.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 15:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 21:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 20:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 20:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 22:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 28 02:38:02 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 17:38:02 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 15:53 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open >Resolution: Accepted Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Nobody/Anonymous (nobody) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 20:38 Message: Logged In: YES user_id=6380 Originator: NO Thanks! Is there anything else that you think needs to be done before I check this in? The core code looks alright to me; I can't be bothered with reviewing the ast stuff or the compiler package since I don't know enough about these, but given that it compiles things correctly I'm not so worried about those. What's the status of the pydoc patch? Are you still working on that? ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-27 20:28 Message: Logged In: YES user_id=24100 Originator: YES Fixed in latest patch. Also added VISIT call for func_annotations. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 19:40 Message: Logged In: YES user_id=6380 Originator: NO I believe I've found a leak in the code that adds annotations to a function object. See this session: >>> x = object() >>> import sys >>> sys.getrefcount(x) 2 >>> for i in range(100): ... def f(x: x): pass ... >>> del f >>> sys.getrefcount(x) 102 >>> At first I thought this could be due to the code added to the MAKE_FUNCTION opcode, but I don't see a leak there. More likely func_annotations is not being freed when a function object is deleted. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 14:05 Message: Logged In: YES user_id=24100 Originator: YES Initial patch to implement keyword-only arguments and annotations support for pydoc and inspect. Tests do not exercise these features, yet. Output for annotations that are types is special cased so that for: def intmin(*a: int) -> int: pass ...help(intmin) will display: intmin(*a: int) -> int File Added: pydoc.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 10:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 16:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 15:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 15:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 17:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 13:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 04:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 03:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 20:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 12:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 20:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 28 02:38:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 17:38:22 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 15:53 Message generated for change (Settings changed) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: Accepted Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) >Assigned to: Guido van Rossum (gvanrossum) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 20:38 Message: Logged In: YES user_id=6380 Originator: NO Thanks! Is there anything else that you think needs to be done before I check this in? The core code looks alright to me; I can't be bothered with reviewing the ast stuff or the compiler package since I don't know enough about these, but given that it compiles things correctly I'm not so worried about those. What's the status of the pydoc patch? Are you still working on that? ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-27 20:28 Message: Logged In: YES user_id=24100 Originator: YES Fixed in latest patch. Also added VISIT call for func_annotations. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 19:40 Message: Logged In: YES user_id=6380 Originator: NO I believe I've found a leak in the code that adds annotations to a function object. See this session: >>> x = object() >>> import sys >>> sys.getrefcount(x) 2 >>> for i in range(100): ... def f(x: x): pass ... >>> del f >>> sys.getrefcount(x) 102 >>> At first I thought this could be due to the code added to the MAKE_FUNCTION opcode, but I don't see a leak there. More likely func_annotations is not being freed when a function object is deleted. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 14:05 Message: Logged In: YES user_id=24100 Originator: YES Initial patch to implement keyword-only arguments and annotations support for pydoc and inspect. Tests do not exercise these features, yet. Output for annotations that are types is special cased so that for: def intmin(*a: int) -> int: pass ...help(intmin) will display: intmin(*a: int) -> int File Added: pydoc.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 10:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 16:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 15:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 15:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 17:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 13:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 04:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 03:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 20:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 12:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 20:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 28 02:39:13 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 17:39:13 -0800 Subject: [Patches] [ python-Patches-1472639 ] make range be xrange Message-ID: Patches item #1472639, was opened at 2006-04-18 18:40 Message generated for change (Comment added) made by gvanrossum You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1472639&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Thomas Wouters (twouters) Assigned to: Neal Norwitz (nnorwitz) Summary: make range be xrange Initial Comment: As discussed (in private email), make range an alias for xrange and see howmuch breaks ;) Aside from Python/bltinmodule.c, 31 files had to be changed: 3 modules (but one of them was a doctest in doctest), 2 test output files (for profile and cProfile), and 26 tests. The predominant breakage in tests was due to tests using range() to express expected-test-output, and comparing it with the output list. Another fair sized portion (particularly of doctests, and including the doctest in doctest itself that had to be updated) broke because of reliance on the repr of range(). Only a few tests broke because of xrange() being immutable (mostly tests that were actually testing list-behaviour, like item- and slice-assignment, on a list created by range()), but that were all two real breakages in actual modules. The only place that broke because xrange can't handle longs was the test for range() that tested whether it'd take longs. Overall, ~185 lines had to be changed. The patch still breaks a test: the test to see if range() does proper checks on its arguments (using the 'badint' class in test_builtin.) I didn't fix it to remind myself that xrange() should be made to operate on longs :) (It currently fails because xrange() will turn all its arguments into C long integers before it does any checks.) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 20:39 Message: Logged In: YES user_id=6380 Originator: NO Ping? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-08-26 16:20 Message: Logged In: YES user_id=6380 For Neal to close when his range/xrange patch is uploaded. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1472639&group_id=5470 From noreply at sourceforge.net Thu Dec 28 04:04:13 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 19:04:13 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 20:53 Message generated for change (Comment added) made by tonylownds You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: Accepted Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Guido van Rossum (gvanrossum) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Tony Lownds (tonylownds) Date: 2006-12-28 03:04 Message: Logged In: YES user_id=24100 Originator: YES Nothing else on the C side of things. The pydoc patch works well for me; more tests ought to be added for function annotations and also for keyword-only arguments, but perhaps that can be added on as a later patch after checkin. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-28 01:38 Message: Logged In: YES user_id=6380 Originator: NO Thanks! Is there anything else that you think needs to be done before I check this in? The core code looks alright to me; I can't be bothered with reviewing the ast stuff or the compiler package since I don't know enough about these, but given that it compiles things correctly I'm not so worried about those. What's the status of the pydoc patch? Are you still working on that? ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-28 01:28 Message: Logged In: YES user_id=24100 Originator: YES Fixed in latest patch. Also added VISIT call for func_annotations. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-28 00:40 Message: Logged In: YES user_id=6380 Originator: NO I believe I've found a leak in the code that adds annotations to a function object. See this session: >>> x = object() >>> import sys >>> sys.getrefcount(x) 2 >>> for i in range(100): ... def f(x: x): pass ... >>> del f >>> sys.getrefcount(x) 102 >>> At first I thought this could be due to the code added to the MAKE_FUNCTION opcode, but I don't see a leak there. More likely func_annotations is not being freed when a function object is deleted. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 19:05 Message: Logged In: YES user_id=24100 Originator: YES Initial patch to implement keyword-only arguments and annotations support for pydoc and inspect. Tests do not exercise these features, yet. Output for annotations that are types is special cased so that for: def intmin(*a: int) -> int: pass ...help(intmin) will display: intmin(*a: int) -> int File Added: pydoc.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 15:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 21:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 20:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 20:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 22:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 18:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 09:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 08:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-20 01:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 17:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-04 01:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 28 07:53:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 27 Dec 2006 22:53:53 -0800 Subject: [Patches] [ python-Patches-1607548 ] Optional Argument Syntax Message-ID: Patches item #1607548, was opened at 2006-12-02 12:53 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 3000 Status: Open Resolution: Accepted Priority: 5 Private: No Submitted By: Tony Lownds (tonylownds) Assigned to: Guido van Rossum (gvanrossum) Summary: Optional Argument Syntax Initial Comment: This patch implements optional argument syntax for Python 3000. The patch still has issues; I am posting so that Collin Winters can add a link to the PEP. The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. This patch alters the MAKE_FUNCTION opcode. I have an implementation that built the func_annotations dictionary in bytecode as well but it was bigger and slower. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-27 22:53 Message: Logged In: YES user_id=33168 Originator: NO I'm skipping the pydoc patch. Didn't even look at it. I don't have the refleak, but I changed some calls and may have fixed it. Committed revision 53170. Leaving open to deal with the pydoc patch. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-27 19:04 Message: Logged In: YES user_id=24100 Originator: YES Nothing else on the C side of things. The pydoc patch works well for me; more tests ought to be added for function annotations and also for keyword-only arguments, but perhaps that can be added on as a later patch after checkin. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 17:38 Message: Logged In: YES user_id=6380 Originator: NO Thanks! Is there anything else that you think needs to be done before I check this in? The core code looks alright to me; I can't be bothered with reviewing the ast stuff or the compiler package since I don't know enough about these, but given that it compiles things correctly I'm not so worried about those. What's the status of the pydoc patch? Are you still working on that? ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-27 17:28 Message: Logged In: YES user_id=24100 Originator: YES Fixed in latest patch. Also added VISIT call for func_annotations. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-27 16:40 Message: Logged In: YES user_id=6380 Originator: NO I believe I've found a leak in the code that adds annotations to a function object. See this session: >>> x = object() >>> import sys >>> sys.getrefcount(x) 2 >>> for i in range(100): ... def f(x: x): pass ... >>> del f >>> sys.getrefcount(x) 102 >>> At first I thought this could be due to the code added to the MAKE_FUNCTION opcode, but I don't see a leak there. More likely func_annotations is not being freed when a function object is deleted. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 11:05 Message: Logged In: YES user_id=24100 Originator: YES Initial patch to implement keyword-only arguments and annotations support for pydoc and inspect. Tests do not exercise these features, yet. Output for annotations that are types is special cased so that for: def intmin(*a: int) -> int: pass ...help(intmin) will display: intmin(*a: int) -> int File Added: pydoc.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-23 07:53 Message: Logged In: YES user_id=24100 Originator: YES Fixed the non-C89 style lines and the formatting (hopefully in compatible style :) File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-22 13:41 Message: Logged In: YES user_id=6380 Originator: NO Thanks for the progress! There are still a few lines ending in whitespace or lines that are longer than 80 chars (and weren't before). Mind cleaning those up? Also ceval.c:2305 and compile.c:1440 contain code that gcc 2.95 won't compile (the 'int' declarations ought to be moved to the start of the containing {...} block); I think this style is not C89 compatible. ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-22 12:15 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Fix crasher in Python/symtable.c -- annotations were visited inside the function scope 2. Fix Lib/compiler issues with Lib/test/test_complex_args. Output from Lib/compiler does not pass all tests, same failures as in HEAD of p3yk branch. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-21 12:21 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Address Neal's comments (I hope) 2. test_scope passes 3. Added some additional tests to test_compiler Open implementation issues: 1. Output from Lib/compiler does not pass test_complex_args, test_scope, possibly more. File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 14:13 Message: Logged In: YES user_id=24100 Originator: YES Changes: 1. Updated to apply cleanly 2. Fix to compile.c so that test_complex_args passes Open implementation issues: 1. Neal's comments 2. test_scope fails 3. Output from Lib/compiler does not pass test_complex_args File Added: opt_arg_ann.patch ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-20 10:04 Message: Logged In: YES user_id=24100 Originator: YES I'll work on code formatting and the error checking and other cleanup. Open to other names than tname and vname, I created those non-terminals in order to use the same code for processing "def" and "lambda". Terminals are caps IIUC. I did add a test for the multi-paren situation. 2.5 had that bug too. Re: no changes to ceval, I tried generating the func_annotations dictionary using bytecodes. That doesn't change the ceval loop but was more code and was slower. So there is a way to avoid ceval changes. Re: deciding if lambda was going to require parens around the arguments, I don't think there was any decision, and yes annotations would be easily supportable. Happy to change if there is support, it's backwards incompatible. Re: return type syntax, I have only seen the -> syntax (vs a keyword 'as') on Guido's blog. Thanks for the comments! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 01:25 Message: Logged In: YES user_id=33168 Originator: NO Nix this comment: I would definitely prefer the annotations baked into the code object so there are no changes to ceval. I see that Guido wants it the way it currently is which makes sense for nested functions. There should probably be a test with nested functions even though it really shouldn't be different. The test will verify that. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-20 00:38 Message: Logged In: YES user_id=33168 Originator: NO When regenerating the patch, can you also remove non-functional changes such as removing unneeded parens and whitespace changes. Also, please try to keep the same formatting in the file wrt tabs and spaces and don't move code around. I know this is a pain and inconsistent. I think I changed ast.c to be all 4 space indents with spaces only. In compiler_simple_arg(), don't you need to check if annotation is NULL when returned from ast_for_expr? Otherwise an undetected error would go through, wouldn't it? In compiler_complex_args(), don't you need to set the ast_error (or a SystemError) if the switch isn't a tname, vname, or LPAR? I don't like the names tname and vname. Also they seem inconsistent. Aren't all the other names all CAPS? In hunk, @@ -602,51 +625,75 @@ remove the commented out code. We shouldn't use any // style comments either. Can you improve the error msg for kwdefaults == NULL? (Thanks for adding it!) Check annotation for NULL if returned from ast_for_expr? BTW, the AST code in this area was tricky code which had some bugs. Did you test with adding extra parentheses and singleton tuples? I'm not sure if Guido preferred syntax -> vs a keyword 'as' for the return type. In symtable.c remove the printfs. They should probably be SystemErrors or something. I would definitely prefer the annotations baked into the code object so there are no changes to ceval. Did we decide if lambda was going to require parens around the arguments? If so, it could support annotations, right? (No comment on the usefulness of annotations for lambdas. :-) In compiler_visit_argannotation, you should return the result from PyList_Append and can remove the comment about checking for errors. Also, I believe the INCREF is not needed, it will be done by PyList_Append. Same deal with returning result of compiler_visit_argannotations() (the one with an s). Need to check for PyList_New() returning NULL in compiler_visit_annotations(). Lots more error checking needs to be added in this area. Dammit, I really want to use Mondrian for these comments! (Sorry Tony, not your fault, I'm just having some bad memories at this point cause I have to keep providing the references.) This patch looks very complete in that it updates things like the compiler package and the parsermodule.c. Good job! This is a great start. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-12-19 17:22 Message: Logged In: YES user_id=6380 Originator: NO Applying the patch fails, probably due to recent merge activities in the p3yk branch. Can I inconvenience you with a request to regenerate the patch from the branch head? ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-11 09:29 Message: Logged In: YES user_id=764593 Originator: NO Could you rename it to "argument annotations"? "optional argument" makes me think of the current keyword arguments, that can be but don't have to be passed. -jJ ---------------------------------------------------------------------- Comment By: Tony Lownds (tonylownds) Date: 2006-12-03 17:24 Message: Logged In: YES user_id=24100 Originator: YES This patch implements optional argument syntax for Python 3000. The patch still has issues: 1. test_ast and test_scope fail. 2. Running the test suite after compiling the library with the compiler package causes failures 3. no docs 4. C-code reference counts and error checking needs a review The syntax implemented is roughly: def f(arg:expr, (nested1:expr, nested2:expr)) -> expr: suite The function object has a new attribute, func_annotations that maps from argument names to the result of the expression. The return annotation is stored with a key of 'return'. Lambda's syntax doesn't support annotations. The ast format has changed for the builtin compiler and the compiler package. A new token was added, '->' (called RARROW in token.h). token.py lost ERRORTOKEN after re-generating, I don't know why. I added it back manually. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1607548&group_id=5470 From noreply at sourceforge.net Thu Dec 28 12:48:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 28 Dec 2006 03:48:07 -0800 Subject: [Patches] [ python-Patches-1623563 ] Allow __class __ assignment for classes with __slots__ Message-ID: Patches item #1623563, was opened at 2006-12-28 11:48 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1623563&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: TH (therve) Assigned to: Nobody/Anonymous (nobody) Summary: Allow __class __ assignment for classes with __slots__ Initial Comment: I made a modification in typeobject.c to allow __class__ modification for classes with slots. It's basically a change in same_slots_added to count the offset of the slots and check if the names of the slots are the same (in the naive way). I don't check if slots are in a different order, that may be an improve. The patch is against trunk, with some tests. It's my first submission on Python, so every feedback will be welcome :). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1623563&group_id=5470 From noreply at sourceforge.net Fri Dec 29 07:01:06 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 28 Dec 2006 22:01:06 -0800 Subject: [Patches] [ python-Patches-1624059 ] fast subclasses of builtin types Message-ID: Patches item #1624059, was opened at 2006-12-28 22:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1624059&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Neal Norwitz (nnorwitz) Assigned to: Guido van Rossum (gvanrossum) Summary: fast subclasses of builtin types Initial Comment: This is similar to a patch posted on python-dev a few months ago (or more). I modified it to also handle subclassing exceptions which should speed up exception handling a bit. (This was proposed by Guido based on the original patch.) I also dropped an extra bit that was going to indicate if it was a builtin type or a subclass of a builtin type. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1624059&group_id=5470 From noreply at sourceforge.net Fri Dec 29 07:04:12 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 28 Dec 2006 22:04:12 -0800 Subject: [Patches] [ python-Patches-1624059 ] fast subclasses of builtin types Message-ID: Patches item #1624059, was opened at 2006-12-28 22:01 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1624059&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Neal Norwitz (nnorwitz) Assigned to: Guido van Rossum (gvanrossum) Summary: fast subclasses of builtin types Initial Comment: This is similar to a patch posted on python-dev a few months ago (or more). I modified it to also handle subclassing exceptions which should speed up exception handling a bit. (This was proposed by Guido based on the original patch.) I also dropped an extra bit that was going to indicate if it was a builtin type or a subclass of a builtin type. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-28 22:04 Message: Logged In: YES user_id=33168 Originator: YES I forgot to mention this patch works by using unused bits in tp_flags. This saves a function call when checking for a subclass of a builtin type. There's one funky thing about this patch, the change to Objects/exceptions.c. I didn't investigate why this was necessary, or more likely I did why when I added it and forgot. I know that without adding BASE_EXC_SUBCLASS to tp_flags, test_exceptions fails. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1624059&group_id=5470 From noreply at sourceforge.net Sat Dec 30 13:11:44 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 30 Dec 2006 04:11:44 -0800 Subject: [Patches] [ python-Patches-1507247 ] tarfile extraction does not honor umask Message-ID: Patches item #1507247, was opened at 2006-06-16 14:11 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Faik Uygur (faik) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile extraction does not honor umask Initial Comment: If the upperdirs in the member file's pathname does not exist. tarfile creates those paths with 0777 permission bits and does not honor umask. This patch uses umask to set the ti.mode of the created directory for later usage in chmod. --- tarfile.py (revision 46993) +++ tarfile.py (working copy) @@ -1560,7 +1560,9 @@ ti = TarInfo() ti.name = upperdirs ti.type = DIRTYPE - ti.mode = 0777 + umask = os.umask(0) + ti.mode = 0777 - umask + os.umask(umask) ti.mtime = tarinfo.mtime ti.uid = tarinfo.uid ti.gid = tarinfo.gid ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-30 13:11 Message: Logged In: YES user_id=642936 Originator: NO In order to determine the current umask we have no other choice AFAIK than to set it with a bogus value, save the return value and restore it right away - as you proposed in your patch. The problem is that there is a small window of time between these two calls where the umask is invalid. This is especially bad in multi-threaded environments. Any ideas? ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-06 00:40 Message: Logged In: YES user_id=161998 Originator: NO Hi, I can reproduce this problem on python 2.4 , and patch applies to python 2.5 too. Fix looks good to me. ---------------------------------------------------------------------- Comment By: Faik Uygur (faik) Date: 2006-08-18 11:44 Message: Logged In: YES user_id=1541018 Above patch is wrong. The correct one is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 From noreply at sourceforge.net Sat Dec 30 13:47:55 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 30 Dec 2006 04:47:55 -0800 Subject: [Patches] [ python-Patches-1616125 ] Cached globals+builtins lookup optimization Message-ID: Patches item #1616125, was opened at 2006-12-15 01:44 Message generated for change (Comment added) made by ag6502 You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Performance Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Andrea Griffini (ag6502) Assigned to: Nobody/Anonymous (nobody) Summary: Cached globals+builtins lookup optimization Initial Comment: This patch implements a speed optimization by introducing a timestamp on dictionaries and by caching lookups of constants so that lookup doesn't need to be repeated if the dictionary didn't change. Currently the patch implements the cached lookup only for the LOAD_GLOBAL opcode and stores the cache as two extra member in the code object. I'm going to investigate if LOAD_ATTR in the special case of a module is worth caching too. A cache entry for LOAD_GLOBAL is defined by two timestamps (Py_ssize_t) and a *borrowed* pointer to a PyObject. The use of a borrowed pointer is safe because the value is used only if the dictionaries didn't change, so the result of lookup must still be alive because the reference in the dictionary that returned the cached value it is still in place. The assumptions are: - a LOAD_GLOBAL opcode always looks for the same symbol (i.e. co_names never changes) - the dictionary being searched (globals and builtins) are mostly constant On my system I got a speedup of 8%-10% on a real program (a tcp/ip server that handles an in-memory database of python objects) on the checkpoint load + rollforward steps (a few millions LOAD_GLOBAL operations in total) and the varying percentage depends on which compiler options are used to build python. On another little test program that calls a function def f(x): return math.sin(x) + math.cos(x) + abs(x) one million times the speedup with the default makefile is about 7%. To enable the patch the makefile should define the CACHED_LOOKUPS symbol. The change to dictobject also exports the timestamp, at the python level; other changes are instead invisible at the python level. I'm not sure at all exporting dict.timestamp() is a good idea however... Note that after applying the patch you may find an error in userdict if you simply use "make" (there's probably some dependency problem somewhere). After make clean + make the test suite runs fine on my system. I'm interested in hearing if also other platforms do actually see a real performance improvement. Andrea ---------------------------------------------------------------------- >Comment By: Andrea Griffini (ag6502) Date: 2006-12-30 13:47 Message: Logged In: YES user_id=1668946 Originator: YES I added the two missing parts and the patch should be now functionally complete; now there co_names reordering pass during compilation so that all names used in LOAD_GLOBAL/LOAD_ATTR are moved at the beginning of the tuple, this allows the lookup cache to be allocated only for the needed size. The reordering is done *before* actually assembling basic blocks to avoid to deal with the change of opcode size after reordering. Also now the timestamp is reset when rolling over to 0 and a sweep is done on all dictionaries and all code objects to clear cached lookups; I didn't implemented it in the gc because: 1) I'm not sure if all code objects and dictionaries are listed (how do I find code objects ?), 2) to do the traversing the code must be in gcmodule, or a richer interface must be exported, 3) what if garbage collecting is disabled ? The extra cost for dictionary creation is normally (i.e. when a dictionary is recovered from the free pool) just setting the timestamp to 1, the extra cost for dictionary update is higher but around the "noise" of timeit.Timer on my system. The memory cost is three more 32-bit words for every dict and 4 + n*3 more words for every code object where n is the number of elements of co_names that are used in LOAD_GLOBAL/LOAD_ATTR opcodes. The patch as it stands don't need to change marshalling of compiled code (the loaded code objects will simply be "unoptimized" so n will be 1+max(oparg) for all arguments of LOAD_GLOBAL/LOAD_ATTR - wasting some cache slots). The patch itself doesn't do anything unless the symbol CACHED_LOOKUPS is defined (this will cache LOAD_GLOBALs) or both CACHED_LOOKUPS and CACHED_MODULE_LOOKUPS are defined (this will cache both LOAD_GLOBALs and LOAD_ATTRs when the searched object is a module). This allows to have this optimization as an optional build option (does this make sense ?). File Added: cached_lookups_12.patch ---------------------------------------------------------------------- Comment By: Andrea Griffini (ag6502) Date: 2006-12-23 23:52 Message: Logged In: YES user_id=1668946 Originator: YES I was already changing the code in the direction you're pointing out. I attached the current version (still not complete!) of the patch; more specifically: - Cached lookups are now stored in an array of structures instead than in two separate arrays - Macros are used for handling of cache entries - The timestamp is now unsigned and there is optional support for 64-bit timestamps - Support for stalling timestamping at 32 bit wrap point - Removed the LOAD_FAST_HACK (was just an experiment with oprofile; was left there by mistake) - Fixed memory allocation failures handling in code object creation - Removed exporting of timestamp value at the python level - Used as cache size the largest LOAD_ATTR/LOAD_GLOBAL entry found Still missing are the sorting of co_names to pack cache slots and the cache reset at gc time if the dictionary timestamp is stalled. The cached lookup can be used at two levels: -DCACHED_LOOKUPS enables caching of LOAD_GLOBAL results -DCACHED_MODULE_LOOKUPS (in addition to -DCACHED_LOOKUPS) also caches the result of a lookup in a module. Speed results are sometimes very interesting and sometimes strange; I found measurable differences just moving around statements between logically equivalent places after checking what gcc was doing with register allocation (probably those places were indeed not equivalent if taking in account aliasing that isn't going to happen but that a c compiler must assume as possible). I have no idea if the speedups I've measured are better or worse on other processors/architectures. File Added: cached_lookups_10.patch ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-12-23 04:33 Message: Logged In: YES user_id=33168 Originator: NO I only reviewed the code in the patch. I have not thought conceptually about this, whether it's a good idea, how to break it, etc. I also did not look to verify all the necessary dict methods were updated to add TOUCHED. Have you profiled the slow down to dicts in normal cases? Things like: d = {} d[0] = 0 etc? All dict operations are going to be a tiny bit slower due to an additional assignment. It's probably not measurable, but you should still verify this. What is LOAD_FAST_HACK in ceval.c? It doesn't seem necessary for this patch. Please revert any whitespace/formatting changes that aren't part of the patch (there are a couple in dictobject.c and codeobject.c). It seems like a little structure would be better than parallel arrays. This would reduce some calculations and reduce the number of mallocs. Also there would only be a single assignment in the eval loop and the names would probably wind up being more clear. Would it be better to use PyObject_Malloc rather than PyMem_Malloc? It should be faster for sizes <= 256. (Despite the name, it doesn't have to handle PyObjects.) Why not use a size_t instead of Py_ssize_t for the timestamp? This is conceptually unsigned, no? Use the macro version PyTuple_GET_SIZE(names); in codeobject.c. It will be a little faster. If the cache allocs fail when allocating a code object, ceval will crash due to not checking for NULL. As for the timestamp, there should probably be a macro to access the field for a dict. This macro should be used in ceval.c/codeobject.c for getting/setting it. When returning the timestamp in dictobject.c you shouldn't cast it to a long. Use the appropriate method such as PyInt_FromSsize_t. ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 23:57 Message: Logged In: YES user_id=21627 Originator: NO I think restricting yourself to 32-bit counters is fine (it's also an issue of memory overhead as you have one of these per access to a global, and per dictionary); you have to deal with the overflow, though. I suggest to use an unsigned value, e.g. size_t. Overflowing should be dealt with through visiting all objects, as you say. It could either happen right when it overflows, or it could get integrated into GC, and only reset when the GC runs. You'd then need to account for those periods when it already has overflowed, but GC hasn't run yet; in these periods, counting should be "on hold". ---------------------------------------------------------------------- Comment By: Andrea Griffini (ag6502) Date: 2006-12-16 18:13 Message: Logged In: YES user_id=1668946 Originator: YES I am experimenting with long long timestamps (based on "unsigned long long" if available or explicitly on two "unsigned long" otherwise). An alternative to slowing down lookups by using 64 bits on 32-bit computers is just using a single 32-bit counter and trigger a "cache flush" when the timestamp rolls over by visiting all live dictionaries and code objects. This would be IMO a *very* infrequent operation (for example on my P4/2.4GHz just doing nothing else but dict updates getting to 2**32 updates would require 14 minutes). ---------------------------------------------------------------------- Comment By: Martin v. L?wis (loewis) Date: 2006-12-16 12:01 Message: Logged In: YES user_id=21627 Originator: NO How does that deal with timestamp wraparound? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1616125&group_id=5470 From noreply at sourceforge.net Sat Dec 30 19:25:00 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 30 Dec 2006 10:25:00 -0800 Subject: [Patches] [ python-Patches-1507247 ] tarfile extraction does not honor umask Message-ID: Patches item #1507247, was opened at 2006-06-16 13:11 Message generated for change (Comment added) made by hanwen You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Faik Uygur (faik) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile extraction does not honor umask Initial Comment: If the upperdirs in the member file's pathname does not exist. tarfile creates those paths with 0777 permission bits and does not honor umask. This patch uses umask to set the ti.mode of the created directory for later usage in chmod. --- tarfile.py (revision 46993) +++ tarfile.py (working copy) @@ -1560,7 +1560,9 @@ ti = TarInfo() ti.name = upperdirs ti.type = DIRTYPE - ti.mode = 0777 + umask = os.umask(0) + ti.mode = 0777 - umask + os.umask(umask) ti.mtime = tarinfo.mtime ti.uid = tarinfo.uid ti.gid = tarinfo.gid ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-30 18:25 Message: Logged In: YES user_id=161998 Originator: NO umask(2) works in the same way, so there seems to be no unixy way to inspect umask without setting it. I think the solution would be to make a C-level function to return the umask (by setting and resetting it). As the interpreter itself is single threaded, this is race-free. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-30 12:11 Message: Logged In: YES user_id=642936 Originator: NO In order to determine the current umask we have no other choice AFAIK than to set it with a bogus value, save the return value and restore it right away - as you proposed in your patch. The problem is that there is a small window of time between these two calls where the umask is invalid. This is especially bad in multi-threaded environments. Any ideas? ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-05 23:40 Message: Logged In: YES user_id=161998 Originator: NO Hi, I can reproduce this problem on python 2.4 , and patch applies to python 2.5 too. Fix looks good to me. ---------------------------------------------------------------------- Comment By: Faik Uygur (faik) Date: 2006-08-18 10:44 Message: Logged In: YES user_id=1541018 Above patch is wrong. The correct one is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 From noreply at sourceforge.net Sun Dec 31 01:13:29 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 30 Dec 2006 16:13:29 -0800 Subject: [Patches] [ python-Patches-1620174 ] Improve platform.py usability on Windows Message-ID: Patches item #1620174, was opened at 2006-12-21 09:49 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Luke Dunstan (infidel) Assigned to: Nobody/Anonymous (nobody) Summary: Improve platform.py usability on Windows Initial Comment: This patch modifies platform.py to remove most of the dependencies on pywin32, and use the standard ctypes and _winreg modules instead. It also adds support for Windows CE. ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-30 19:13 Message: Logged In: YES user_id=764593 Originator: NO ( win32api.RegQueryValueEx is _winreg.QueryValueEx ) ? If not, it should wait for 2.6, and there should be an entry in what's new. (I suppose similar concerns exist for other return classes.) The change to win32_ver only half-corrects the return type to the four-tuple. The meaning of release (even if it is just "release name") should be specified in the text. def win32_ver(release='',version='',csd='',ptype=''): """ Get additional version information from the Windows Registry - and return a tuple (version,csd,ptype) referring to version + and return a tuple (release,version,csd,ptype) referring to version number, CSD level and OS type (multi/single processor). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470 From noreply at sourceforge.net Sun Dec 31 01:26:29 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 30 Dec 2006 16:26:29 -0800 Subject: [Patches] [ python-Patches-1623563 ] Allow __class __ assignment for classes with __slots__ Message-ID: Patches item #1623563, was opened at 2006-12-28 06:48 Message generated for change (Comment added) made by jimjjewett You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1623563&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.6 Status: Open Resolution: None Priority: 5 Private: No Submitted By: TH (therve) Assigned to: Nobody/Anonymous (nobody) Summary: Allow __class __ assignment for classes with __slots__ Initial Comment: I made a modification in typeobject.c to allow __class__ modification for classes with slots. It's basically a change in same_slots_added to count the offset of the slots and check if the names of the slots are the same (in the naive way). I don't check if slots are in a different order, that may be an improve. The patch is against trunk, with some tests. It's my first submission on Python, so every feedback will be welcome :). ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-30 19:26 Message: Logged In: YES user_id=764593 Originator: NO Just going through the list doesn't seem so naive to me. If the slots are in a different order, then you would need to move the data around -- which borders on "maybe they ought to have written the translator themselves" ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1623563&group_id=5470 From noreply at sourceforge.net Sun Dec 31 06:57:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 30 Dec 2006 21:57:07 -0800 Subject: [Patches] [ python-Patches-1620174 ] Improve platform.py usability on Windows Message-ID: Patches item #1620174, was opened at 2006-12-21 22:49 Message generated for change (Comment added) made by infidel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Luke Dunstan (infidel) Assigned to: Nobody/Anonymous (nobody) Summary: Improve platform.py usability on Windows Initial Comment: This patch modifies platform.py to remove most of the dependencies on pywin32, and use the standard ctypes and _winreg modules instead. It also adds support for Windows CE. ---------------------------------------------------------------------- >Comment By: Luke Dunstan (infidel) Date: 2006-12-31 13:57 Message: Logged In: YES user_id=30442 Originator: YES 1. Yes this is intended for 2.6 2. The only difference between win32api.RegQueryValueEx and _winreg.QueryValueEx seems to be that the latter returns Unicode strings. I have adjusted the patch to be more compatible with the old behaviour. 3. I have updated the doc string in the new patch. File Added: platform-wince-2.diff ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-31 08:13 Message: Logged In: YES user_id=764593 Originator: NO ( win32api.RegQueryValueEx is _winreg.QueryValueEx ) ? If not, it should wait for 2.6, and there should be an entry in what's new. (I suppose similar concerns exist for other return classes.) The change to win32_ver only half-corrects the return type to the four-tuple. The meaning of release (even if it is just "release name") should be specified in the text. def win32_ver(release='',version='',csd='',ptype=''): """ Get additional version information from the Windows Registry - and return a tuple (version,csd,ptype) referring to version + and return a tuple (release,version,csd,ptype) referring to version number, CSD level and OS type (multi/single processor). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470 From noreply at sourceforge.net Sun Dec 31 12:52:05 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 31 Dec 2006 03:52:05 -0800 Subject: [Patches] [ python-Patches-1507247 ] tarfile extraction does not honor umask Message-ID: Patches item #1507247, was opened at 2006-06-16 14:11 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Private: No Submitted By: Faik Uygur (faik) Assigned to: Lars Gust?bel (gustaebel) Summary: tarfile extraction does not honor umask Initial Comment: If the upperdirs in the member file's pathname does not exist. tarfile creates those paths with 0777 permission bits and does not honor umask. This patch uses umask to set the ti.mode of the created directory for later usage in chmod. --- tarfile.py (revision 46993) +++ tarfile.py (working copy) @@ -1560,7 +1560,9 @@ ti = TarInfo() ti.name = upperdirs ti.type = DIRTYPE - ti.mode = 0777 + umask = os.umask(0) + ti.mode = 0777 - umask + os.umask(umask) ti.mtime = tarinfo.mtime ti.uid = tarinfo.uid ti.gid = tarinfo.gid ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-31 12:52 Message: Logged In: YES user_id=642936 Originator: NO I've come to the conclusion that it is a doubtful approach to take the mtime and ownership from the file and use it on the upper directories as well. So, I've come up with a totally different solution (cp. makedirs.diff) that abandons the use of os.umask() completely and uses a single call to os.makedirs() to create the missing directories. It seems very attractive to me to do it this way, what do you think? File Added: makedirs.diff ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-30 19:25 Message: Logged In: YES user_id=161998 Originator: NO umask(2) works in the same way, so there seems to be no unixy way to inspect umask without setting it. I think the solution would be to make a C-level function to return the umask (by setting and resetting it). As the interpreter itself is single threaded, this is race-free. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-12-30 13:11 Message: Logged In: YES user_id=642936 Originator: NO In order to determine the current umask we have no other choice AFAIK than to set it with a bogus value, save the return value and restore it right away - as you proposed in your patch. The problem is that there is a small window of time between these two calls where the umask is invalid. This is especially bad in multi-threaded environments. Any ideas? ---------------------------------------------------------------------- Comment By: Han-Wen Nienhuys (hanwen) Date: 2006-12-06 00:40 Message: Logged In: YES user_id=161998 Originator: NO Hi, I can reproduce this problem on python 2.4 , and patch applies to python 2.5 too. Fix looks good to me. ---------------------------------------------------------------------- Comment By: Faik Uygur (faik) Date: 2006-08-18 11:44 Message: Logged In: YES user_id=1541018 Above patch is wrong. The correct one is attached. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1507247&group_id=5470 From noreply at sourceforge.net Sun Dec 31 19:49:57 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 31 Dec 2006 10:49:57 -0800 Subject: [Patches] [ python-Patches-1620174 ] Improve platform.py usability on Windows Message-ID: Patches item #1620174, was opened at 2006-12-21 15:49 Message generated for change (Comment added) made by lemburg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None Status: Open Resolution: None Priority: 5 Private: No Submitted By: Luke Dunstan (infidel) >Assigned to: M.-A. Lemburg (lemburg) Summary: Improve platform.py usability on Windows Initial Comment: This patch modifies platform.py to remove most of the dependencies on pywin32, and use the standard ctypes and _winreg modules instead. It also adds support for Windows CE. ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2006-12-31 19:49 Message: Logged In: YES user_id=38388 Originator: NO I haven't looked at the patch yet, so just a few general comments on changes to platform.py: * the code must continue to work with Python versions prior to 2.6 This means that ctypes and _winreg support may be added as an option, but removing pywin32 calls is not the right way to proceed. * changes in return type of the public and documented APIs are not possible If you have a need for more information, then a new API should be added, or the information merged into one of the existing return fields. * changes in the return values of APIs due to use of different OS APIs must be avoided There's code out there relying on the return values, so if in doubt a new API must be provided. ---------------------------------------------------------------------- Comment By: Luke Dunstan (infidel) Date: 2006-12-31 06:57 Message: Logged In: YES user_id=30442 Originator: YES 1. Yes this is intended for 2.6 2. The only difference between win32api.RegQueryValueEx and _winreg.QueryValueEx seems to be that the latter returns Unicode strings. I have adjusted the patch to be more compatible with the old behaviour. 3. I have updated the doc string in the new patch. File Added: platform-wince-2.diff ---------------------------------------------------------------------- Comment By: Jim Jewett (jimjjewett) Date: 2006-12-31 01:13 Message: Logged In: YES user_id=764593 Originator: NO ( win32api.RegQueryValueEx is _winreg.QueryValueEx ) ? If not, it should wait for 2.6, and there should be an entry in what's new. (I suppose similar concerns exist for other return classes.) The change to win32_ver only half-corrects the return type to the four-tuple. The meaning of release (even if it is just "release name") should be specified in the text. def win32_ver(release='',version='',csd='',ptype=''): """ Get additional version information from the Windows Registry - and return a tuple (version,csd,ptype) referring to version + and return a tuple (release,version,csd,ptype) referring to version number, CSD level and OS type (multi/single processor). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1620174&group_id=5470