From noreply@sourceforge.net Thu Nov 1 00:31:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 16:31:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-472568 ] PyBuffer_New() memory not aligned Message-ID: Bugs item #472568, was opened at 2001-10-18 14:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: PyBuffer_New() memory not aligned Initial Comment: Memory buffer areas created by PyBuffer_New are missaligned on 32 bit machines, for doubles (64 bit). This typically generates a bus error crash on most RISC machines when a library function tries to write a double in a memory buffer allocated by PyBuffer_New(). When looking at the bufferobject.c code, this seems to come from the fact that the memory buffer points at 'malloc() + sizeof struct PyBufferObject'; [line: b->b_ptr = (void *)(b + 1); ] To the extent that struct PyBufferObject is not a multiple of sizeof (double), the largest value type; thus the misalignment, and the crashes on most RISC processors. A quick and temporary solution would consist in adding a dummy double field as the last field of the PyBufferObject structure: struct PyBufferObject { ... double dummy; } ; and setting b->b_ptr = (void*)&b->dummy; /*was (void*(b+1)*/ In doing so, b->b_ptr will always by aligned on sizeof (double), on either 32 and 64 bit architectures. Since I'm on the buffer type problem: It would be nice (and probably easy to do) to augment the buffer protocol interface with the ability to specify the basic value type stored in the buffer. A possible list of values (enum ...) would be: - undefined (backward compatibility) - char, unsigned char, short, ....int, ... long, long long, float, double, and void* (memory address). This would enable to check at runtime the type of values stored in a buffer (and prevent missalignement buserrors, as well as catching garbage situations when improper array types are passed by means of the buffer interface [e.g.: int/float/double/short...). Frederic Giacometti Frederic Giacometti ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-31 16:31 Message: Logged In: YES user_id=93657 A portable solution (im[provement over what I proposed) would consitst in declaring 'dummy' with a union type, 'unionizing' all C-ANSI value types (and including 'long long' optionally by mean of an #ifdef). { .... union { int Int; long Long; double Double; void* Pvoid ...} dummy; } All (void*)obj->dummy can be replaced with obj->dummy.Pvoid FG ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-22 13:19 Message: Logged In: YES user_id=31392 Note to whomever take this bug: PyBuffer_New() is not called anywhere in the Python source tree; nor are there any tests for buffer objects that I'm aware of. A few simple test cases would have caught this bug already. (And for the case of the builtin buffer() call, it might be good if it used PyBuffer_New().) ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-18 14:35 Message: Logged In: YES user_id=93657 I wasn't looged in when I submitted the item. Don't think I'm becoming anonymous :)) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 From noreply@sourceforge.net Thu Nov 1 00:41:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 16:41:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-477023 ] pdb: unexpected path confuses Emacs Message-ID: Bugs item #477023, was opened at 2001-10-31 16:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477023&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Mark (jhmark) Assigned to: Nobody/Anonymous (nobody) Summary: pdb: unexpected path confuses Emacs Initial Comment: With python 2.2b1, Start the debugger like this on any script. Notice the stack frame display in which has a path name prepended to it. This confuses the emacs Python debugging mode (python-mode.el) causing it not to recognize the line as a stack frame, and thus not to display the (Pdb) prompt for the user. $ ./bin/python ./lib/python2.2/pdb.py ~/tmp/foo.py > /hda3/sys/Python-2.2b1-local/(0)?() (Pdb) in the same scenario, python 1.5 does not confuse emacs because it displays the line like this: > (0)?() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477023&group_id=5470 From noreply@sourceforge.net Thu Nov 1 01:31:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 17:31:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-472568 ] PyBuffer_New() memory not aligned Message-ID: Bugs item #472568, was opened at 2001-10-18 14:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: PyBuffer_New() memory not aligned Initial Comment: Memory buffer areas created by PyBuffer_New are missaligned on 32 bit machines, for doubles (64 bit). This typically generates a bus error crash on most RISC machines when a library function tries to write a double in a memory buffer allocated by PyBuffer_New(). When looking at the bufferobject.c code, this seems to come from the fact that the memory buffer points at 'malloc() + sizeof struct PyBufferObject'; [line: b->b_ptr = (void *)(b + 1); ] To the extent that struct PyBufferObject is not a multiple of sizeof (double), the largest value type; thus the misalignment, and the crashes on most RISC processors. A quick and temporary solution would consist in adding a dummy double field as the last field of the PyBufferObject structure: struct PyBufferObject { ... double dummy; } ; and setting b->b_ptr = (void*)&b->dummy; /*was (void*(b+1)*/ In doing so, b->b_ptr will always by aligned on sizeof (double), on either 32 and 64 bit architectures. Since I'm on the buffer type problem: It would be nice (and probably easy to do) to augment the buffer protocol interface with the ability to specify the basic value type stored in the buffer. A possible list of values (enum ...) would be: - undefined (backward compatibility) - char, unsigned char, short, ....int, ... long, long long, float, double, and void* (memory address). This would enable to check at runtime the type of values stored in a buffer (and prevent missalignement buserrors, as well as catching garbage situations when improper array types are passed by means of the buffer interface [e.g.: int/float/double/short...). Frederic Giacometti Frederic Giacometti ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-31 17:31 Message: Logged In: YES user_id=31435 The only portable way to fix this (assuming it's broken -- I don't see any alignment promises in the docs, and since we never use it we can't mine the source code for clues either) is to have PyBuffer_New do a second separate malloc (size) and set b_ptr to that. The C std guarantees memory returned by malloc is suitably aligned for all purposes; it doesn't promise that any standard type captures the strictest alignment requirement (indeed, at KSR we returned 128-byte aligned memory from malloc, to cater to our "subpage" type extension). ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-31 16:31 Message: Logged In: YES user_id=93657 A portable solution (im[provement over what I proposed) would consitst in declaring 'dummy' with a union type, 'unionizing' all C-ANSI value types (and including 'long long' optionally by mean of an #ifdef). { .... union { int Int; long Long; double Double; void* Pvoid ...} dummy; } All (void*)obj->dummy can be replaced with obj->dummy.Pvoid FG ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-22 13:19 Message: Logged In: YES user_id=31392 Note to whomever take this bug: PyBuffer_New() is not called anywhere in the Python source tree; nor are there any tests for buffer objects that I'm aware of. A few simple test cases would have caught this bug already. (And for the case of the builtin buffer() call, it might be good if it used PyBuffer_New().) ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-18 14:35 Message: Logged In: YES user_id=93657 I wasn't looged in when I submitted the item. Don't think I'm becoming anonymous :)) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 From noreply@sourceforge.net Thu Nov 1 02:07:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 18:07:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-476912 ] regex annoyance Message-ID: Bugs item #476912, was opened at 2001-10-31 12:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 >Category: Regular Expressions Group: None Status: Open Resolution: None Priority: 5 Submitted By: Bill Bumgarner (bbum) >Assigned to: Fredrik Lundh (effbot) Summary: regex annoyance Initial Comment: (this may be a feature request-- but it is annoying enough that I filed it as a bug) Python's named sub expressions within regular expressions are an incredibly valuable feature; between it and the ability to automatically collapse multiline regex's w/comments leads to very readable regex's. However, there is an annoyance in named subexpressions that has bitten me several times. Namely, if you have a situation where a particular token must be parsed out of the input through the use of one of two (or more) expressions in a fashion that cannot be expressed without multiple possible means of matching any given subexpression, then the named subexpression will only be non-None intermittently (depending on expression order and what was matched). That is, given: (?:(?[a-z]+)\s(?[a-z]+))|(?:(? [a-z]+)\t(?[a-z]+)) In this case, Tok1 and Tok2 will be None if the first expression matches... (Yes, this is a contrived example that could be refactored to not use multiple / references-- however, more complex expressions do not always enable easy refactoring.) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-31 18:07 Message: Logged In: YES user_id=31435 Since symbolic names are names *of* integer group numbers, the regexp compiler should really raise an exception when seeing a given symbolic name defined more than once in a regexp. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 From noreply@sourceforge.net Thu Nov 1 02:48:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 18:48:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-476912 ] regex annoyance Message-ID: Bugs item #476912, was opened at 2001-10-31 12:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 Category: Regular Expressions Group: None Status: Open Resolution: None Priority: 5 Submitted By: Bill Bumgarner (bbum) Assigned to: Fredrik Lundh (effbot) Summary: regex annoyance Initial Comment: (this may be a feature request-- but it is annoying enough that I filed it as a bug) Python's named sub expressions within regular expressions are an incredibly valuable feature; between it and the ability to automatically collapse multiline regex's w/comments leads to very readable regex's. However, there is an annoyance in named subexpressions that has bitten me several times. Namely, if you have a situation where a particular token must be parsed out of the input through the use of one of two (or more) expressions in a fashion that cannot be expressed without multiple possible means of matching any given subexpression, then the named subexpression will only be non-None intermittently (depending on expression order and what was matched). That is, given: (?:(?[a-z]+)\s(?[a-z]+))|(?:(? [a-z]+)\t(?[a-z]+)) In this case, Tok1 and Tok2 will be None if the first expression matches... (Yes, this is a contrived example that could be refactored to not use multiple / references-- however, more complex expressions do not always enable easy refactoring.) ---------------------------------------------------------------------- >Comment By: Bill Bumgarner (bbum) Date: 2001-10-31 18:48 Message: Logged In: YES user_id=103811 While I agree that the proposed solution of raising an exception would certainly be more acceptable behavior than what is occurring now, doing away with support for multiple subexpressions with the same name would be undesirable. In particular, named subexpressions allow the developer to decouple oneself from counting expressions. It also allows the developer to not fall into a situation where they have to write a few lines of if/else statements to get the value when it might be in either expression A or expression B. I would rather an error be raised if two separate instances of named expression A were both defined. As long as only one matches, then it shouldn't matter that it appears twice. The goal is to be able to do this|that where this and that both define the same set of named subexpressions. By definition, only one of this or that will match and, therefore, only one value could be had for a named expression that appears in both this and that. (As it stands, I have numerous lines of if/else 'this or that' code that generally causes clutter. It means that the groupdict() cannot be treated as a pure result-- I often have to go through the this/that logic to normalize the groupdict into something that actually represents the results I desired). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 18:07 Message: Logged In: YES user_id=31435 Since symbolic names are names *of* integer group numbers, the regexp compiler should really raise an exception when seeing a given symbolic name defined more than once in a regexp. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 From noreply@sourceforge.net Thu Nov 1 03:53:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 19:53:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-477023 ] pdb: unexpected path confuses Emacs Message-ID: Bugs item #477023, was opened at 2001-10-31 16:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477023&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jonathan Mark (jhmark) Assigned to: Nobody/Anonymous (nobody) Summary: pdb: unexpected path confuses Emacs Initial Comment: With python 2.2b1, Start the debugger like this on any script. Notice the stack frame display in which has a path name prepended to it. This confuses the emacs Python debugging mode (python-mode.el) causing it not to recognize the line as a stack frame, and thus not to display the (Pdb) prompt for the user. $ ./bin/python ./lib/python2.2/pdb.py ~/tmp/foo.py > /hda3/sys/Python-2.2b1-local/(0)?() (Pdb) in the same scenario, python 1.5 does not confuse emacs because it displays the line like this: > (0)?() ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-31 19:53 Message: Logged In: YES user_id=6380 Here's a patch to bdb.py that might fix this. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477023&group_id=5470 From noreply@sourceforge.net Thu Nov 1 06:45:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 22:45:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-476912 ] regex annoyance Message-ID: Bugs item #476912, was opened at 2001-10-31 12:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 Category: Regular Expressions Group: None Status: Open Resolution: None Priority: 5 Submitted By: Bill Bumgarner (bbum) Assigned to: Fredrik Lundh (effbot) Summary: regex annoyance Initial Comment: (this may be a feature request-- but it is annoying enough that I filed it as a bug) Python's named sub expressions within regular expressions are an incredibly valuable feature; between it and the ability to automatically collapse multiline regex's w/comments leads to very readable regex's. However, there is an annoyance in named subexpressions that has bitten me several times. Namely, if you have a situation where a particular token must be parsed out of the input through the use of one of two (or more) expressions in a fashion that cannot be expressed without multiple possible means of matching any given subexpression, then the named subexpression will only be non-None intermittently (depending on expression order and what was matched). That is, given: (?:(?[a-z]+)\s(?[a-z]+))|(?:(? [a-z]+)\t(?[a-z]+)) In this case, Tok1 and Tok2 will be None if the first expression matches... (Yes, this is a contrived example that could be refactored to not use multiple / references-- however, more complex expressions do not always enable easy refactoring.) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-10-31 22:45 Message: Logged In: YES user_id=31435 Bill, you misunderstand my comment: I'm not trying to solve your problem . Named groups were my idea to begin with (years ago), and what you want of them is both unclear and beyond their intended use. I'm not suggesting to take *away* "support for multiple subexpressions with the same name": there is no such support, only the illusion of support due to the regexp compiler failing to raise an exception when a name is redefined (that's an old bug, btw: it's persisted across three generations of underlying regexp engine). Group names are nothing but synonyms for numbered groups; they add no power, just convenience. If you want more than that, that's fine, but then you need to specify exactly what happens in all cases, and get that implemented. The semantics of named groups right now are defined in terms of a trivial bijection with numbered groups, and all you're seeing when you repeat a name is implementation accidents due to a failure to enforce that there *is* a bijection. ---------------------------------------------------------------------- Comment By: Bill Bumgarner (bbum) Date: 2001-10-31 18:48 Message: Logged In: YES user_id=103811 While I agree that the proposed solution of raising an exception would certainly be more acceptable behavior than what is occurring now, doing away with support for multiple subexpressions with the same name would be undesirable. In particular, named subexpressions allow the developer to decouple oneself from counting expressions. It also allows the developer to not fall into a situation where they have to write a few lines of if/else statements to get the value when it might be in either expression A or expression B. I would rather an error be raised if two separate instances of named expression A were both defined. As long as only one matches, then it shouldn't matter that it appears twice. The goal is to be able to do this|that where this and that both define the same set of named subexpressions. By definition, only one of this or that will match and, therefore, only one value could be had for a named expression that appears in both this and that. (As it stands, I have numerous lines of if/else 'this or that' code that generally causes clutter. It means that the groupdict() cannot be treated as a pure result-- I often have to go through the this/that logic to normalize the groupdict into something that actually represents the results I desired). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 18:07 Message: Logged In: YES user_id=31435 Since symbolic names are names *of* integer group numbers, the regexp compiler should really raise an exception when seeing a given symbolic name defined more than once in a regexp. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 From noreply@sourceforge.net Thu Nov 1 07:50:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 31 Oct 2001 23:50:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-476912 ] regex annoyance Message-ID: Bugs item #476912, was opened at 2001-10-31 12:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 Category: Regular Expressions Group: None Status: Open >Resolution: Fixed Priority: 5 Submitted By: Bill Bumgarner (bbum) Assigned to: Fredrik Lundh (effbot) Summary: regex annoyance Initial Comment: (this may be a feature request-- but it is annoying enough that I filed it as a bug) Python's named sub expressions within regular expressions are an incredibly valuable feature; between it and the ability to automatically collapse multiline regex's w/comments leads to very readable regex's. However, there is an annoyance in named subexpressions that has bitten me several times. Namely, if you have a situation where a particular token must be parsed out of the input through the use of one of two (or more) expressions in a fashion that cannot be expressed without multiple possible means of matching any given subexpression, then the named subexpression will only be non-None intermittently (depending on expression order and what was matched). That is, given: (?:(?[a-z]+)\s(?[a-z]+))|(?:(? [a-z]+)\t(?[a-z]+)) In this case, Tok1 and Tok2 will be None if the first expression matches... (Yes, this is a contrived example that could be refactored to not use multiple / references-- however, more complex expressions do not always enable easy refactoring.) ---------------------------------------------------------------------- >Comment By: Fredrik Lundh (effbot) Date: 2001-10-31 23:50 Message: Logged In: YES user_id=38376 This will be fixed (as in "explicitly disallowed") in 2.2b2. (but I guess it's time to start thinking about building a better framework on top of SRE. after all, the engine itself can do what Bill wants...) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 22:45 Message: Logged In: YES user_id=31435 Bill, you misunderstand my comment: I'm not trying to solve your problem . Named groups were my idea to begin with (years ago), and what you want of them is both unclear and beyond their intended use. I'm not suggesting to take *away* "support for multiple subexpressions with the same name": there is no such support, only the illusion of support due to the regexp compiler failing to raise an exception when a name is redefined (that's an old bug, btw: it's persisted across three generations of underlying regexp engine). Group names are nothing but synonyms for numbered groups; they add no power, just convenience. If you want more than that, that's fine, but then you need to specify exactly what happens in all cases, and get that implemented. The semantics of named groups right now are defined in terms of a trivial bijection with numbered groups, and all you're seeing when you repeat a name is implementation accidents due to a failure to enforce that there *is* a bijection. ---------------------------------------------------------------------- Comment By: Bill Bumgarner (bbum) Date: 2001-10-31 18:48 Message: Logged In: YES user_id=103811 While I agree that the proposed solution of raising an exception would certainly be more acceptable behavior than what is occurring now, doing away with support for multiple subexpressions with the same name would be undesirable. In particular, named subexpressions allow the developer to decouple oneself from counting expressions. It also allows the developer to not fall into a situation where they have to write a few lines of if/else statements to get the value when it might be in either expression A or expression B. I would rather an error be raised if two separate instances of named expression A were both defined. As long as only one matches, then it shouldn't matter that it appears twice. The goal is to be able to do this|that where this and that both define the same set of named subexpressions. By definition, only one of this or that will match and, therefore, only one value could be had for a named expression that appears in both this and that. (As it stands, I have numerous lines of if/else 'this or that' code that generally causes clutter. It means that the groupdict() cannot be treated as a pure result-- I often have to go through the this/that logic to normalize the groupdict into something that actually represents the results I desired). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 18:07 Message: Logged In: YES user_id=31435 Since symbolic names are names *of* integer group numbers, the regexp compiler should really raise an exception when seeing a given symbolic name defined more than once in a regexp. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 From noreply@sourceforge.net Thu Nov 1 14:22:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 06:22:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: Mark Rowe (icep17) Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 06:22 Message: Logged In: YES user_id=21627 I cannot reproduce this in 2.2b1 (it works the same on Win32 as it does on Linux), but I also fail to see what code change may have caused to fail it in 2.2a3. Please update to 2.2b1 and retest; if it still happens, check that you don't have any extra copies of python22.dll. If so, please add a note in this report, and we'll re-open it. If you can, it would help enourmously if you could run the program in a debugger and report the stack trace in case of error, and the file and line number causing the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Thu Nov 1 15:41:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 07:41:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 07:41 Message: Logged In: NO gcc version is 2.95.3 HP-UX hp18 B.11.11 U 9000/785 2011589119 both Python 2.1.1 and 2.2b1 do not compile ootb here. Error messages during compile are building '_socket' extension gcc -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/utmnt/ut/si/dv0216/Python-2.2b1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.2/socketmodule.o In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:42, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: `__va__list' redefined /usr/include/sys/stdsyms.h:422: warning: this is the location of the previous definition In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:47, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/string.h:26: warning: `__va__list' redefined /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: this is the location of the previous definition In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:152: /usr/include/sys/socket.h:439: parse error before `sendfile' /usr/include/sys/socket.h:439: parse error before `bsize_t' /usr/include/sys/socket.h:440: warning: data definition has no type or storage class /usr/include/sys/socket.h:441: parse error before `sendpath' /usr/include/sys/socket.h:441: parse error before `bsize_t' /usr/include/sys/socket.h:442: warning: data definition has no type or storage class /usr/include/sys/socket.h:456: parse error before `__sendfile64' /usr/include/sys/socket.h:456: parse error before `bsize_t' /usr/include/sys/socket.h:456: warning: data definition has no type or storage class /usr/include/sys/socket.h:457: parse error before `__sendpath64' /usr/include/sys/socket.h:457: parse error before `bsize_t' /usr/include/sys/socket.h:457: warning: data definition has no type or storage class /usr/include/sys/socket.h:459: parse error before `sendfile' /usr/include/sys/socket.h:459: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendfile': /usr/include/sys/socket.h:459: parse error before `bsize_t' /usr/include/sys/socket.h: At top level: /usr/include/sys/socket.h:460: parse error before `sendpath' /usr/include/sys/socket.h:460: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendpath': /usr/include/sys/socket.h:460: parse error before `bsize_t' In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:238: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: At top level: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:203: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:212: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `freeaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:219: warning: implicit declaration of function `free' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `str_isnumber': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:231: warning: subscript has type `char' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `getaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:345: warning: implicit declaration of function `atoi' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:396: warning: implicit declaration of function `malloc' WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 I now thought that since the errors happen in a system header that compiles with cc and not gcc, why not copy to "socket.h" and delete the two offending lines. However, as you see, is still included (why ?). Another option is using cc just for this library, but I'm not sure, that the object files are compatible. Using cc alone creates a lot of non-working modules, although it is in (extended) ANSI mode. I also tried the HP Porting centre (http://hpux.asknet.de) but the Python 2.1 package there -just like mine- does crash on importing socket (too with 2.2b1) hp18: Python-2.2b1 % ./python Python 2.2b1 (#4, Nov 1 2001, 16:36:54) [C] on hp-uxB Type "help", "copyright", "credits" or "license" for more information. >>> import socket Traceback (most recent call last): File "", line 1, in ? File "/utmnt/ut/si/dv0216/Python-2.2b1/Lib/socket.py", line 41, in ? from _socket import * ImportError: No module named _socket >>> ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-30 00:56 Message: Logged In: YES user_id=21627 It would be helpful if you would provide all details in a report also. When you say "it crashes", did you really mean "it crashes" (i.e. with a core dump, machine reboot, or the like)? If the compiler merely rejected the code, it would be good to see what the error message was. Please also report version numbers: HP-UX version, gcc version, etc. Are you sure you are using a gcc built for your OS version? If the gcc doesn't get prototypes right, this often comes from having fixincluded header files of a different OS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 04:59 Message: Logged In: YES user_id=6380 Please try again with Python 2.2b1 (http://python.org/2.2/). We've done a lot of work and we've got at least one report that it now works out of the box on HP-UX 11. I'd like to hear if that solves your problems (and if 2.2b1 builds with either compiler). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Thu Nov 1 16:57:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 08:57:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 08:57 Message: Logged In: YES user_id=21627 gcc 2.95 does not support HP-UX 11; this appears to be an instance of that problem. The __va__list problem is not at all Python-specific; it occurs in many packages (just search Google for "HP-UX __va__list"). With such problems, there is no point into looking further into the problem; just drop the compiler. If you want to port Python to HP-UX and gcc 2.95, you are on your own. Notice that gcc 3.0.1 *does* support HP-UX 11; I recommend to upgrade. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 07:41 Message: Logged In: NO gcc version is 2.95.3 HP-UX hp18 B.11.11 U 9000/785 2011589119 both Python 2.1.1 and 2.2b1 do not compile ootb here. Error messages during compile are building '_socket' extension gcc -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/utmnt/ut/si/dv0216/Python-2.2b1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.2/socketmodule.o In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:42, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: `__va__list' redefined /usr/include/sys/stdsyms.h:422: warning: this is the location of the previous definition In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:47, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/string.h:26: warning: `__va__list' redefined /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: this is the location of the previous definition In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:152: /usr/include/sys/socket.h:439: parse error before `sendfile' /usr/include/sys/socket.h:439: parse error before `bsize_t' /usr/include/sys/socket.h:440: warning: data definition has no type or storage class /usr/include/sys/socket.h:441: parse error before `sendpath' /usr/include/sys/socket.h:441: parse error before `bsize_t' /usr/include/sys/socket.h:442: warning: data definition has no type or storage class /usr/include/sys/socket.h:456: parse error before `__sendfile64' /usr/include/sys/socket.h:456: parse error before `bsize_t' /usr/include/sys/socket.h:456: warning: data definition has no type or storage class /usr/include/sys/socket.h:457: parse error before `__sendpath64' /usr/include/sys/socket.h:457: parse error before `bsize_t' /usr/include/sys/socket.h:457: warning: data definition has no type or storage class /usr/include/sys/socket.h:459: parse error before `sendfile' /usr/include/sys/socket.h:459: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendfile': /usr/include/sys/socket.h:459: parse error before `bsize_t' /usr/include/sys/socket.h: At top level: /usr/include/sys/socket.h:460: parse error before `sendpath' /usr/include/sys/socket.h:460: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendpath': /usr/include/sys/socket.h:460: parse error before `bsize_t' In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:238: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: At top level: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:203: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:212: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `freeaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:219: warning: implicit declaration of function `free' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `str_isnumber': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:231: warning: subscript has type `char' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `getaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:345: warning: implicit declaration of function `atoi' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:396: warning: implicit declaration of function `malloc' WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 I now thought that since the errors happen in a system header that compiles with cc and not gcc, why not copy to "socket.h" and delete the two offending lines. However, as you see, is still included (why ?). Another option is using cc just for this library, but I'm not sure, that the object files are compatible. Using cc alone creates a lot of non-working modules, although it is in (extended) ANSI mode. I also tried the HP Porting centre (http://hpux.asknet.de) but the Python 2.1 package there -just like mine- does crash on importing socket (too with 2.2b1) hp18: Python-2.2b1 % ./python Python 2.2b1 (#4, Nov 1 2001, 16:36:54) [C] on hp-uxB Type "help", "copyright", "credits" or "license" for more information. >>> import socket Traceback (most recent call last): File "", line 1, in ? File "/utmnt/ut/si/dv0216/Python-2.2b1/Lib/socket.py", line 41, in ? from _socket import * ImportError: No module named _socket >>> ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-30 00:56 Message: Logged In: YES user_id=21627 It would be helpful if you would provide all details in a report also. When you say "it crashes", did you really mean "it crashes" (i.e. with a core dump, machine reboot, or the like)? If the compiler merely rejected the code, it would be good to see what the error message was. Please also report version numbers: HP-UX version, gcc version, etc. Are you sure you are using a gcc built for your OS version? If the gcc doesn't get prototypes right, this often comes from having fixincluded header files of a different OS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 04:59 Message: Logged In: YES user_id=6380 Please try again with Python 2.2b1 (http://python.org/2.2/). We've done a lot of work and we've got at least one report that it now works out of the box on HP-UX 11. I'd like to hear if that solves your problems (and if 2.2b1 builds with either compiler). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Thu Nov 1 17:21:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 09:21:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-472568 ] PyBuffer_New() memory not aligned Message-ID: Bugs item #472568, was opened at 2001-10-18 14:31 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: PyBuffer_New() memory not aligned Initial Comment: Memory buffer areas created by PyBuffer_New are missaligned on 32 bit machines, for doubles (64 bit). This typically generates a bus error crash on most RISC machines when a library function tries to write a double in a memory buffer allocated by PyBuffer_New(). When looking at the bufferobject.c code, this seems to come from the fact that the memory buffer points at 'malloc() + sizeof struct PyBufferObject'; [line: b->b_ptr = (void *)(b + 1); ] To the extent that struct PyBufferObject is not a multiple of sizeof (double), the largest value type; thus the misalignment, and the crashes on most RISC processors. A quick and temporary solution would consist in adding a dummy double field as the last field of the PyBufferObject structure: struct PyBufferObject { ... double dummy; } ; and setting b->b_ptr = (void*)&b->dummy; /*was (void*(b+1)*/ In doing so, b->b_ptr will always by aligned on sizeof (double), on either 32 and 64 bit architectures. Since I'm on the buffer type problem: It would be nice (and probably easy to do) to augment the buffer protocol interface with the ability to specify the basic value type stored in the buffer. A possible list of values (enum ...) would be: - undefined (backward compatibility) - char, unsigned char, short, ....int, ... long, long long, float, double, and void* (memory address). This would enable to check at runtime the type of values stored in a buffer (and prevent missalignement buserrors, as well as catching garbage situations when improper array types are passed by means of the buffer interface [e.g.: int/float/double/short...). Frederic Giacometti Frederic Giacometti ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-11-01 09:21 Message: Logged In: YES user_id=93657 The solution Tim proposes (2d malloc) will be very fine with what we do. Here are some more details on what we're doing (and this is a standard operation): - We want to create an array of double that we pass to a C function, and then return this array to Python as buffer object (the buffer is passed latter on as arg to other functions using the buffer interface); and do it so that Python takes ownership of the buffer memory management. - We don't want to require Numerical to operate the package; just for memory allocation. - We should actually be using the Python array module interface. Unfortunately: * the Python array object C definitions are not exported in a .h file * the Array python interface does not provide the ability to create a new array of an arbitrary size (and certainly initialized to 0). One has to provide a list or a string to create an array of a given size. IThis is not workable if we work we large arrays (e.g.: an array of 1.000.000 doubles is only 8 MB RAM ...). Another solution, then, would consist in extending the array Python interface, so as to enable the creation of arrays of arbitrary sizes (prefereably initialized to 0 or to something alse with a calloc or a memset). The extension of the array.array() function could be the better solution, taking into account our needs as well as Tim's concerns. FG ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 17:31 Message: Logged In: YES user_id=31435 The only portable way to fix this (assuming it's broken -- I don't see any alignment promises in the docs, and since we never use it we can't mine the source code for clues either) is to have PyBuffer_New do a second separate malloc (size) and set b_ptr to that. The C std guarantees memory returned by malloc is suitably aligned for all purposes; it doesn't promise that any standard type captures the strictest alignment requirement (indeed, at KSR we returned 128-byte aligned memory from malloc, to cater to our "subpage" type extension). ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-31 16:31 Message: Logged In: YES user_id=93657 A portable solution (im[provement over what I proposed) would consitst in declaring 'dummy' with a union type, 'unionizing' all C-ANSI value types (and including 'long long' optionally by mean of an #ifdef). { .... union { int Int; long Long; double Double; void* Pvoid ...} dummy; } All (void*)obj->dummy can be replaced with obj->dummy.Pvoid FG ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-22 13:19 Message: Logged In: YES user_id=31392 Note to whomever take this bug: PyBuffer_New() is not called anywhere in the Python source tree; nor are there any tests for buffer objects that I'm aware of. A few simple test cases would have caught this bug already. (And for the case of the builtin buffer() call, it might be good if it used PyBuffer_New().) ---------------------------------------------------------------------- Comment By: Frederic Giacometti (giacometti) Date: 2001-10-18 14:35 Message: Logged In: YES user_id=93657 I wasn't looged in when I submitted the item. Don't think I'm becoming anonymous :)) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472568&group_id=5470 From noreply@sourceforge.net Thu Nov 1 17:57:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 09:57:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-477215 ] 'in' and 'not in' won't work for dict Message-ID: Bugs item #477215, was opened at 2001-11-01 09:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Kerim Borchaev (warkid) Assigned to: Nobody/Anonymous (nobody) Summary: 'in' and 'not in' won't work for dict Initial Comment: In spite of what docs say. (They say : "Equivalent to a.has_key(k)") but: >>> 0 in {} Traceback (most recent call last): File "", line 1, in ? TypeError: 'in' or 'not in' needs sequence right argument ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 From noreply@sourceforge.net Thu Nov 1 18:40:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 10:40:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-477221 ] abs and divmod act oddly with -0.0 Message-ID: Bugs item #477221, was opened at 2001-11-01 10:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: abs and divmod act oddly with -0.0 Initial Comment: Python 2.1.1 (#97, Aug 2 2001, 21:53:31) [CW PPC GUSI2 THREADS] on mac Type "copyright", "credits" or "license" for more information. >>> abs(-1.0) 1.0 >>> abs(-0.0) -0.0 >>> divmod(-0.0, 1.0) (0.0, -0.0) The same thing happens on a unix Python 2.0, so I think it's pretty generic. I hope folks think it's a bug, not a feature, and worth fixing. It bit me when trying to write a sexagesimal formatter -- output that should have shown -0:00:00 showed 0:00:-0 instead. -- Russell Russell Owen owen@astro.washington.edu ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 From noreply@sourceforge.net Thu Nov 1 19:11:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 11:11:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-477215 ] 'in' and 'not in' won't work for dict Message-ID: Bugs item #477215, was opened at 2001-11-01 09:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Kerim Borchaev (warkid) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: 'in' and 'not in' won't work for dict Initial Comment: In spite of what docs say. (They say : "Equivalent to a.has_key(k)") but: >>> 0 in {} Traceback (most recent call last): File "", line 1, in ? TypeError: 'in' or 'not in' needs sequence right argument ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-01 11:11 Message: Logged In: YES user_id=31435 Which version of Python are you using? This doesn't work in any released final version of Python -- it's a new-in- 2.2 feature (which hasn't been released yet). Assigning to Fred because I seem to recall that the docs for this snuck out early by mistake. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 From noreply@sourceforge.net Thu Nov 1 19:16:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 11:16:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-477221 ] abs and divmod act oddly with -0.0 Message-ID: Bugs item #477221, was opened at 2001-11-01 10:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: abs and divmod act oddly with -0.0 Initial Comment: Python 2.1.1 (#97, Aug 2 2001, 21:53:31) [CW PPC GUSI2 THREADS] on mac Type "copyright", "credits" or "license" for more information. >>> abs(-1.0) 1.0 >>> abs(-0.0) -0.0 >>> divmod(-0.0, 1.0) (0.0, -0.0) The same thing happens on a unix Python 2.0, so I think it's pretty generic. I hope folks think it's a bug, not a feature, and worth fixing. It bit me when trying to write a sexagesimal formatter -- output that should have shown -0:00:00 showed 0:00:-0 instead. -- Russell Russell Owen owen@astro.washington.edu ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-01 11:16 Message: Logged In: YES user_id=31435 A bit of a dilemma here. Python doesn't know anything about signed zeroes, and there's really no portable way in C to detect one when it happens (until C99 is universally implemented). Even that you're getting "-0.0" output is a platform-dependent accident (it depends on what the platform C sprintf does; and, e.g., on Windows it does not display a leading minus sign). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 From noreply@sourceforge.net Thu Nov 1 19:32:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 11:32:28 -0800 Subject: [Python-bugs-list] [ python-Bugs-477215 ] 'in' and 'not in' won't work for dict Message-ID: Bugs item #477215, was opened at 2001-11-01 09:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Kerim Borchaev (warkid) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: 'in' and 'not in' won't work for dict Initial Comment: In spite of what docs say. (They say : "Equivalent to a.has_key(k)") but: >>> 0 in {} Traceback (most recent call last): File "", line 1, in ? TypeError: 'in' or 'not in' needs sequence right argument ---------------------------------------------------------------------- >Comment By: Kerim Borchaev (warkid) Date: 2001-11-01 11:32 Message: Logged In: YES user_id=314933 yes it's about 2.1.1. And it seems that it is in the docs for a while... ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 11:11 Message: Logged In: YES user_id=31435 Which version of Python are you using? This doesn't work in any released final version of Python -- it's a new-in- 2.2 feature (which hasn't been released yet). Assigning to Fred because I seem to recall that the docs for this snuck out early by mistake. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 From noreply@sourceforge.net Thu Nov 1 19:38:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 11:38:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-477215 ] 'in' and 'not in' won't work for dict Message-ID: Bugs item #477215, was opened at 2001-11-01 09:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 Category: None Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Kerim Borchaev (warkid) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: 'in' and 'not in' won't work for dict Initial Comment: In spite of what docs say. (They say : "Equivalent to a.has_key(k)") but: >>> 0 in {} Traceback (most recent call last): File "", line 1, in ? TypeError: 'in' or 'not in' needs sequence right argument ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-01 11:38 Message: Logged In: YES user_id=3066 This is definately a doc bug; the online version of the docs has been fixed. The documentation for Python 2.1.2 has the required correction; that will be available later this year. ---------------------------------------------------------------------- Comment By: Kerim Borchaev (warkid) Date: 2001-11-01 11:32 Message: Logged In: YES user_id=314933 yes it's about 2.1.1. And it seems that it is in the docs for a while... ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 11:11 Message: Logged In: YES user_id=31435 Which version of Python are you using? This doesn't work in any released final version of Python -- it's a new-in- 2.2 feature (which hasn't been released yet). Assigning to Fred because I seem to recall that the docs for this snuck out early by mistake. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477215&group_id=5470 From noreply@sourceforge.net Thu Nov 1 19:50:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 11:50:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-476898 ] Word "separate" spelled incorrectly Message-ID: Bugs item #476898, was opened at 2001-10-31 11:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 1 Submitted By: David J. Ritchie (r124c41) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Word "separate" spelled incorrectly Initial Comment: The word "separate" is spelled incorrectly in Section 1.6.1 of "Macintosh Library Modules". See "file in a seperate script window" in: http://www.python.org/doc/current/mac/node15.html I have not looked at other sections of the document. It may be elsewhere in the document as well. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-01 11:50 Message: Logged In: YES user_id=3066 Fixed in Doc/mac/using.tex revision 1.4 and 1.3.2.1; two occurances. ---------------------------------------------------------------------- Comment By: David J. Ritchie (r124c41) Date: 2001-10-31 11:35 Message: Logged In: YES user_id=364368 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 From noreply@sourceforge.net Thu Nov 1 20:11:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 12:11:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-477221 ] abs and divmod act oddly with -0.0 Message-ID: Bugs item #477221, was opened at 2001-11-01 10:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: abs and divmod act oddly with -0.0 Initial Comment: Python 2.1.1 (#97, Aug 2 2001, 21:53:31) [CW PPC GUSI2 THREADS] on mac Type "copyright", "credits" or "license" for more information. >>> abs(-1.0) 1.0 >>> abs(-0.0) -0.0 >>> divmod(-0.0, 1.0) (0.0, -0.0) The same thing happens on a unix Python 2.0, so I think it's pretty generic. I hope folks think it's a bug, not a feature, and worth fixing. It bit me when trying to write a sexagesimal formatter -- output that should have shown -0:00:00 showed 0:00:-0 instead. -- Russell Russell Owen owen@astro.washington.edu ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-01 12:11 Message: Logged In: YES user_id=31435 abs(-0.0) returns +0.0, as of Objects/floatobject.c; new revision: 2.103 Haven't looked at divmod yet. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 11:16 Message: Logged In: YES user_id=31435 A bit of a dilemma here. Python doesn't know anything about signed zeroes, and there's really no portable way in C to detect one when it happens (until C99 is universally implemented). Even that you're getting "-0.0" output is a platform-dependent accident (it depends on what the platform C sprintf does; and, e.g., on Windows it does not display a leading minus sign). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 From noreply@sourceforge.net Thu Nov 1 20:37:30 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 12:37:30 -0800 Subject: [Python-bugs-list] [ python-Bugs-476419 ] Turtle documentation not included. Message-ID: Bugs item #476419, was opened at 2001-10-30 09:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476419&group_id=5470 Category: Documentation Group: None Status: Open >Resolution: Later >Priority: 3 Submitted By: Josh Cogliati (jjc) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Turtle documentation not included. Initial Comment: Problem: The file libturtle.tex has been written. However it is not included in lib.tex Also in libundoc.tex the turtle library is listed as undocumented, dispite the fact that documentation has been written. Solution: Include libturtle.tex in lib.tex and remove the references to turtle being undocumented in libundoc.tex ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-01 12:37 Message: Logged In: YES user_id=3066 While this is seemingly easy to fix, the real problem (which is why the turtle docs have never been included), is there there hasn't been a good place to put it (no GUI or Tkinter chapter). Fortunately, this is being corrected, so we should see this section included with Python 2.2 (probably in beta 2). Dropping the priority since it will be taken care of as part of an existing effort. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476419&group_id=5470 From noreply@sourceforge.net Thu Nov 1 21:52:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 13:52:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-477221 ] abs and divmod act oddly with -0.0 Message-ID: Bugs item #477221, was opened at 2001-11-01 10:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: abs and divmod act oddly with -0.0 Initial Comment: Python 2.1.1 (#97, Aug 2 2001, 21:53:31) [CW PPC GUSI2 THREADS] on mac Type "copyright", "credits" or "license" for more information. >>> abs(-1.0) 1.0 >>> abs(-0.0) -0.0 >>> divmod(-0.0, 1.0) (0.0, -0.0) The same thing happens on a unix Python 2.0, so I think it's pretty generic. I hope folks think it's a bug, not a feature, and worth fixing. It bit me when trying to write a sexagesimal formatter -- output that should have shown -0:00:00 showed 0:00:-0 instead. -- Russell Russell Owen owen@astro.washington.edu ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-01 13:52 Message: Logged In: YES user_id=31435 And abs(float) was fixed again , in Objects/floatobject.c new revision: 2.104 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 12:11 Message: Logged In: YES user_id=31435 abs(-0.0) returns +0.0, as of Objects/floatobject.c; new revision: 2.103 Haven't looked at divmod yet. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 11:16 Message: Logged In: YES user_id=31435 A bit of a dilemma here. Python doesn't know anything about signed zeroes, and there's really no portable way in C to detect one when it happens (until C99 is universally implemented). Even that you're getting "-0.0" output is a platform-dependent accident (it depends on what the platform C sprintf does; and, e.g., on Windows it does not display a leading minus sign). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 From noreply@sourceforge.net Thu Nov 1 23:15:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 15:15:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-477221 ] abs and divmod act oddly with -0.0 Message-ID: Bugs item #477221, was opened at 2001-11-01 10:40 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: abs and divmod act oddly with -0.0 Initial Comment: Python 2.1.1 (#97, Aug 2 2001, 21:53:31) [CW PPC GUSI2 THREADS] on mac Type "copyright", "credits" or "license" for more information. >>> abs(-1.0) 1.0 >>> abs(-0.0) -0.0 >>> divmod(-0.0, 1.0) (0.0, -0.0) The same thing happens on a unix Python 2.0, so I think it's pretty generic. I hope folks think it's a bug, not a feature, and worth fixing. It bit me when trying to write a sexagesimal formatter -- output that should have shown -0:00:00 showed 0:00:-0 instead. -- Russell Russell Owen owen@astro.washington.edu ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-01 15:15 Message: Logged In: YES user_id=31435 divmod(-0., 1.) should return (-0.0, +0.0) across platforms now, although the minus sign may not *display* across platforms (does not on Windows, probably does on most Unix flavors). This was something of a nightmare, as it turned out the underlying libm fmod() gives different results across platforms in the presence of signed zeroes. Objects/floatobject.c; new revision: 2.105 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 13:52 Message: Logged In: YES user_id=31435 And abs(float) was fixed again , in Objects/floatobject.c new revision: 2.104 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 12:11 Message: Logged In: YES user_id=31435 abs(-0.0) returns +0.0, as of Objects/floatobject.c; new revision: 2.103 Haven't looked at divmod yet. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 11:16 Message: Logged In: YES user_id=31435 A bit of a dilemma here. Python doesn't know anything about signed zeroes, and there's really no portable way in C to detect one when it happens (until C99 is universally implemented). Even that you're getting "-0.0" output is a platform-dependent accident (it depends on what the platform C sprintf does; and, e.g., on Windows it does not display a leading minus sign). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477221&group_id=5470 From noreply@sourceforge.net Thu Nov 1 23:18:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 15:18:12 -0800 Subject: [Python-bugs-list] [ python-Bugs-476904 ] Unquittable applets Message-ID: Bugs item #476904, was opened at 2001-10-31 11:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476904&group_id=5470 Category: Macintosh Group: Platform-specific >Status: Closed Resolution: None Priority: 5 Submitted By: Just van Rossum (jvr) Assigned to: Jack Jansen (jackjansen) Summary: Unquittable applets Initial Comment: MacPython applets which have the "disable console until needed" option set cannot quit upon error exit because there is no quit menu (not even a file menu...). ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-11-01 15:18 Message: Logged In: YES user_id=45365 Fixed in the repository. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476904&group_id=5470 From noreply@sourceforge.net Fri Nov 2 03:22:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 19:22:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-477371 ] build_scripts can use wrong #! line Message-ID: Bugs item #477371, was opened at 2001-11-01 19:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477371&group_id=5470 Category: Distutils Group: None Status: Open Resolution: None Priority: 5 Submitted By: Matthew Mueller (donut) Assigned to: A.M. Kuchling (akuchling) Summary: build_scripts can use wrong #! line Initial Comment: When the build_scripts command is run, it places into build/scripts a copy of the script with the #! line modified to call the python version directly. But if you subsequently call setup.py with a different python version, the script is not updated to use the correct python executable path. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477371&group_id=5470 From noreply@sourceforge.net Fri Nov 2 03:34:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 19:34:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-476898 ] Word "separate" spelled incorrectly Message-ID: Bugs item #476898, was opened at 2001-10-31 11:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 Category: Documentation Group: None Status: Closed Resolution: Fixed Priority: 1 Submitted By: David J. Ritchie (r124c41) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: Word "separate" spelled incorrectly Initial Comment: The word "separate" is spelled incorrectly in Section 1.6.1 of "Macintosh Library Modules". See "file in a seperate script window" in: http://www.python.org/doc/current/mac/node15.html I have not looked at other sections of the document. It may be elsewhere in the document as well. ---------------------------------------------------------------------- >Comment By: David J. Ritchie (r124c41) Date: 2001-11-01 19:34 Message: Logged In: YES user_id=364368 Thanks it is such a good document that I hated to let the 'blem' go by. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-01 11:50 Message: Logged In: YES user_id=3066 Fixed in Doc/mac/using.tex revision 1.4 and 1.3.2.1; two occurances. ---------------------------------------------------------------------- Comment By: David J. Ritchie (r124c41) Date: 2001-10-31 11:35 Message: Logged In: YES user_id=364368 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476898&group_id=5470 From noreply@sourceforge.net Fri Nov 2 04:05:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 20:05:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Works For Me Priority: 5 Submitted By: Mark Rowe (icep17) Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- >Comment By: Mark Rowe (icep17) Date: 2001-11-01 20:05 Message: Logged In: YES user_id=147838 Under Windows ME with Python 2.2b1 I still get the crash. I have asked my friend to check it (Windows 95c Python 2.2b1) and he also gets the crash. I compiled the debug build of Python and running the above code in an interpreter session gives: Adding parser accelerators ... Done. Python 2.2b1 (#25, Oct 21 2001, 13:42:02) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket [8983 refs] >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) C:\Program Files\python22\Python-2.2b1\Objects\tupleobject.c:147 negative ref co unt -572662308 Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple [9020 refs] >>> The Debug build of Python does not crash, but instead reports a negative refrence count, which could be the cause of the crash in the release build. As I am not too familiar with the internals of Python I have trouble understanding what is happening when running the interpreter under the debugger, but if you would like some more detail I will happily provide it. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 06:22 Message: Logged In: YES user_id=21627 I cannot reproduce this in 2.2b1 (it works the same on Win32 as it does on Linux), but I also fail to see what code change may have caused to fail it in 2.2a3. Please update to 2.2b1 and retest; if it still happens, check that you don't have any extra copies of python22.dll. If so, please add a note in this report, and we'll re-open it. If you can, it would help enourmously if you could run the program in a debugger and report the stack trace in case of error, and the file and line number causing the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Fri Nov 2 04:15:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 20:15:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Works For Me Priority: 5 Submitted By: Mark Rowe (icep17) Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 20:15 Message: Logged In: NO I can reproduce it with 2.2b1 on Win9x, and by using that same code. ---------------------------------------------------------------------- Comment By: Mark Rowe (icep17) Date: 2001-11-01 20:05 Message: Logged In: YES user_id=147838 Under Windows ME with Python 2.2b1 I still get the crash. I have asked my friend to check it (Windows 95c Python 2.2b1) and he also gets the crash. I compiled the debug build of Python and running the above code in an interpreter session gives: Adding parser accelerators ... Done. Python 2.2b1 (#25, Oct 21 2001, 13:42:02) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket [8983 refs] >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) C:\Program Files\python22\Python-2.2b1\Objects\tupleobject.c:147 negative ref co unt -572662308 Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple [9020 refs] >>> The Debug build of Python does not crash, but instead reports a negative refrence count, which could be the cause of the crash in the release build. As I am not too familiar with the internals of Python I have trouble understanding what is happening when running the interpreter under the debugger, but if you would like some more detail I will happily provide it. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 06:22 Message: Logged In: YES user_id=21627 I cannot reproduce this in 2.2b1 (it works the same on Win32 as it does on Linux), but I also fail to see what code change may have caused to fail it in 2.2a3. Please update to 2.2b1 and retest; if it still happens, check that you don't have any extra copies of python22.dll. If so, please add a note in this report, and we'll re-open it. If you can, it would help enourmously if you could run the program in a debugger and report the stack trace in case of error, and the file and line number causing the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Fri Nov 2 05:24:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 01 Nov 2001 21:24:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None >Status: Open Resolution: Works For Me >Priority: 7 Submitted By: Mark Rowe (icep17) Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-01 21:24 Message: Logged In: YES user_id=31435 Re-opened this and raised priority. Martin, near the start of PySocket_getnameinfo we've got n = PyArg_ParseTuple(sa, "si|ii", &hostp, &port, &flowinfo, scope_id); and that's surely incorrect (note that we're not passing the address of scope_id there! we're passing its value, and scope_id is uninitialized stack trash). There's more than just that going on here, though. If that's repaired, the PyArg_ParseTuple fails, and so it jumps to the "fail:" label, and sa gets decref'ed there. But sa was obtained from a previous PyArg_ParseTuple call, so should not be decref'ed by this routine (this routine has a borrowed reference to sa). That part almost certainly explains the "negative refcount" msg icep17 sees in a debug build. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 20:15 Message: Logged In: NO I can reproduce it with 2.2b1 on Win9x, and by using that same code. ---------------------------------------------------------------------- Comment By: Mark Rowe (icep17) Date: 2001-11-01 20:05 Message: Logged In: YES user_id=147838 Under Windows ME with Python 2.2b1 I still get the crash. I have asked my friend to check it (Windows 95c Python 2.2b1) and he also gets the crash. I compiled the debug build of Python and running the above code in an interpreter session gives: Adding parser accelerators ... Done. Python 2.2b1 (#25, Oct 21 2001, 13:42:02) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket [8983 refs] >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) C:\Program Files\python22\Python-2.2b1\Objects\tupleobject.c:147 negative ref co unt -572662308 Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple [9020 refs] >>> The Debug build of Python does not crash, but instead reports a negative refrence count, which could be the cause of the crash in the release build. As I am not too familiar with the internals of Python I have trouble understanding what is happening when running the interpreter under the debugger, but if you would like some more detail I will happily provide it. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 06:22 Message: Logged In: YES user_id=21627 I cannot reproduce this in 2.2b1 (it works the same on Win32 as it does on Linux), but I also fail to see what code change may have caused to fail it in 2.2a3. Please update to 2.2b1 and retest; if it still happens, check that you don't have any extra copies of python22.dll. If so, please add a note in this report, and we'll re-open it. If you can, it would help enourmously if you could run the program in a debugger and report the stack trace in case of error, and the file and line number causing the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Fri Nov 2 09:05:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 01:05:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None Status: Open Resolution: Works For Me Priority: 7 Submitted By: Mark Rowe (icep17) Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-02 01:05 Message: Logged In: YES user_id=21627 Mark, Thanks for your report. It seems Tim is right on all accounts, so there is no need for you to gather further information. I'll try to come up with a scenario where the crash is more reproducible, and check in a fix shortly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 21:24 Message: Logged In: YES user_id=31435 Re-opened this and raised priority. Martin, near the start of PySocket_getnameinfo we've got n = PyArg_ParseTuple(sa, "si|ii", &hostp, &port, &flowinfo, scope_id); and that's surely incorrect (note that we're not passing the address of scope_id there! we're passing its value, and scope_id is uninitialized stack trash). There's more than just that going on here, though. If that's repaired, the PyArg_ParseTuple fails, and so it jumps to the "fail:" label, and sa gets decref'ed there. But sa was obtained from a previous PyArg_ParseTuple call, so should not be decref'ed by this routine (this routine has a borrowed reference to sa). That part almost certainly explains the "negative refcount" msg icep17 sees in a debug build. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 20:15 Message: Logged In: NO I can reproduce it with 2.2b1 on Win9x, and by using that same code. ---------------------------------------------------------------------- Comment By: Mark Rowe (icep17) Date: 2001-11-01 20:05 Message: Logged In: YES user_id=147838 Under Windows ME with Python 2.2b1 I still get the crash. I have asked my friend to check it (Windows 95c Python 2.2b1) and he also gets the crash. I compiled the debug build of Python and running the above code in an interpreter session gives: Adding parser accelerators ... Done. Python 2.2b1 (#25, Oct 21 2001, 13:42:02) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket [8983 refs] >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) C:\Program Files\python22\Python-2.2b1\Objects\tupleobject.c:147 negative ref co unt -572662308 Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple [9020 refs] >>> The Debug build of Python does not crash, but instead reports a negative refrence count, which could be the cause of the crash in the release build. As I am not too familiar with the internals of Python I have trouble understanding what is happening when running the interpreter under the debugger, but if you would like some more detail I will happily provide it. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 06:22 Message: Logged In: YES user_id=21627 I cannot reproduce this in 2.2b1 (it works the same on Win32 as it does on Linux), but I also fail to see what code change may have caused to fail it in 2.2a3. Please update to 2.2b1 and retest; if it still happens, check that you don't have any extra copies of python22.dll. If so, please add a note in this report, and we'll re-open it. If you can, it would help enourmously if you could run the program in a debugger and report the stack trace in case of error, and the file and line number causing the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Fri Nov 2 11:03:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 03:03:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-477487 ] Build Problems on AIX 4.3.3 Message-ID: Bugs item #477487, was opened at 2001-11-02 03:03 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477487&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Build Problems on AIX 4.3.3 Initial Comment: Problems in building Python 2.2b1 in AIX 4.3.3 with IBM vac++ compiler. I think the problem is not compiler specific. The build tries to execute the following code: ../Modules/makexp_aix Modules/python.exp "" libpython2.2.a; -Wl,-bE:Modules/python.exp -lld -o python \ Modules/ccpython.o \ libpython2.2.a -ldl -lpthread -lm The first part executes ok: ../Modules/makexp_aix Modules/python.exp "" libpython2.2.a In the second part the call to the linker is missing. Something like : xlC_r -Wl,-bE:Modules/python.exp -lld -o python Modules/ccpython.o libpython2.2.a -ldl -lpthread -lm dose the job ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477487&group_id=5470 From noreply@sourceforge.net Fri Nov 2 12:15:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 04:15:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-474831 ] Command history doesn't work on Mandrake Message-ID: Bugs item #474831, was opened at 2001-10-25 04:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 Category: Python Interpreter Core Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: peter nordlund (qnorpet) Assigned to: Guido van Rossum (gvanrossum) Summary: Command history doesn't work on Mandrake Initial Comment: When I build version 2.2b1 and 2.2a4 on my Linux Mandrake 8.1 pc the command history doesn't work. The python that is included in Mandrake 8.1 works ok in that regard. I can't use arrow keys or ctrl-p to go backwards. No ctrl- or arrow key-strokes seems to work. When I build 2.2a4 for Solaris it works ok. I have Swedish keyboards, in case that could matter. Best regards, Peter Nordlund ---------------------------------------------------------------------- >Comment By: peter nordlund (qnorpet) Date: 2001-11-02 04:15 Message: Logged In: YES user_id=358954 After following montanaros suggestion to install the -devel rpm, things worked out of the box! /Peter ---------------------------------------------------------------------- Comment By: peter nordlund (qnorpet) Date: 2001-10-26 03:52 Message: Logged In: YES user_id=358954 Ok, before I saw montanaros suggestion I built readline4.2 (I think it was) with --with-termcap, and edited Setup as follows: readline readline.c -I/work1/pkg/utils/readline/4.2/include -L/work1/pkg/utils/readline/4.2/lib -L/lib -lreadline - ltermcap I think I also had mo make a symlink from /lib/libtermcap.so.2.0.8 to /lib/termcap.so I could be wrong in some details, since I'm not sitting right at the Linux-box right now. Then it worked as it should!!! I will try monataros suggestion and will report back after the weekend. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-10-25 08:15 Message: Logged In: YES user_id=44345 Actually, you may well have the readline RPM installed, but not the readline-devel RPM. To compile programs that include support for a given package, you need to install the "-devel" RPM, which includes the various .h files needed to compile the program. Python's setup.py script is probably looking for, but not finding /usr/include/readline/readline.h. Skip ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-25 05:56 Message: Logged In: YES user_id=6380 You probably don't have the GNU readline library installed in a place where Python looks for it. Download its source from a GNU site and build and install it. (If you *do* have GNU readline installed, it would be interesting to find out why setup.py doesn't find it and/or why it doesn't build the readline extension module.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474831&group_id=5470 From noreply@sourceforge.net Fri Nov 2 14:01:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 06:01:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-476912 ] regex annoyance Message-ID: Bugs item #476912, was opened at 2001-10-31 12:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 Category: Regular Expressions Group: None >Status: Closed Resolution: Fixed Priority: 5 Submitted By: Bill Bumgarner (bbum) Assigned to: Fredrik Lundh (effbot) Summary: regex annoyance Initial Comment: (this may be a feature request-- but it is annoying enough that I filed it as a bug) Python's named sub expressions within regular expressions are an incredibly valuable feature; between it and the ability to automatically collapse multiline regex's w/comments leads to very readable regex's. However, there is an annoyance in named subexpressions that has bitten me several times. Namely, if you have a situation where a particular token must be parsed out of the input through the use of one of two (or more) expressions in a fashion that cannot be expressed without multiple possible means of matching any given subexpression, then the named subexpression will only be non-None intermittently (depending on expression order and what was matched). That is, given: (?:(?[a-z]+)\s(?[a-z]+))|(?:(? [a-z]+)\t(?[a-z]+)) In this case, Tok1 and Tok2 will be None if the first expression matches... (Yes, this is a contrived example that could be refactored to not use multiple / references-- however, more complex expressions do not always enable easy refactoring.) ---------------------------------------------------------------------- Comment By: Fredrik Lundh (effbot) Date: 2001-10-31 23:50 Message: Logged In: YES user_id=38376 This will be fixed (as in "explicitly disallowed") in 2.2b2. (but I guess it's time to start thinking about building a better framework on top of SRE. after all, the engine itself can do what Bill wants...) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 22:45 Message: Logged In: YES user_id=31435 Bill, you misunderstand my comment: I'm not trying to solve your problem . Named groups were my idea to begin with (years ago), and what you want of them is both unclear and beyond their intended use. I'm not suggesting to take *away* "support for multiple subexpressions with the same name": there is no such support, only the illusion of support due to the regexp compiler failing to raise an exception when a name is redefined (that's an old bug, btw: it's persisted across three generations of underlying regexp engine). Group names are nothing but synonyms for numbered groups; they add no power, just convenience. If you want more than that, that's fine, but then you need to specify exactly what happens in all cases, and get that implemented. The semantics of named groups right now are defined in terms of a trivial bijection with numbered groups, and all you're seeing when you repeat a name is implementation accidents due to a failure to enforce that there *is* a bijection. ---------------------------------------------------------------------- Comment By: Bill Bumgarner (bbum) Date: 2001-10-31 18:48 Message: Logged In: YES user_id=103811 While I agree that the proposed solution of raising an exception would certainly be more acceptable behavior than what is occurring now, doing away with support for multiple subexpressions with the same name would be undesirable. In particular, named subexpressions allow the developer to decouple oneself from counting expressions. It also allows the developer to not fall into a situation where they have to write a few lines of if/else statements to get the value when it might be in either expression A or expression B. I would rather an error be raised if two separate instances of named expression A were both defined. As long as only one matches, then it shouldn't matter that it appears twice. The goal is to be able to do this|that where this and that both define the same set of named subexpressions. By definition, only one of this or that will match and, therefore, only one value could be had for a named expression that appears in both this and that. (As it stands, I have numerous lines of if/else 'this or that' code that generally causes clutter. It means that the groupdict() cannot be treated as a pure result-- I often have to go through the this/that logic to normalize the groupdict into something that actually represents the results I desired). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-31 18:07 Message: Logged In: YES user_id=31435 Since symbolic names are names *of* integer group numbers, the regexp compiler should really raise an exception when seeing a given symbolic name defined more than once in a regexp. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476912&group_id=5470 From noreply@sourceforge.net Fri Nov 2 16:23:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 08:23:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-477557 ] email package needs examples Message-ID: Bugs item #477557, was opened at 2001-11-02 08:23 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477557&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: email package needs examples Initial Comment: Fill out the examples section of the email package documentation. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477557&group_id=5470 From noreply@sourceforge.net Fri Nov 2 17:20:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 09:20:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None Status: Open Resolution: Works For Me Priority: 7 Submitted By: Mark Rowe (icep17) Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-02 09:20 Message: Logged In: YES user_id=31435 Martin, I was wrong about one thing: scope_id isn't uninitialized stack trash, it's forced to 0. So one way to reliably provoke the first problem is simply to pass an argument with a value for scope_id (then the 0 passed for scope_id looks like a NULL pointer to PyArg_ParseTuple). Like, say, >>> socket.getnameinfo(('x', 0, 0, 0), 0) Boom. I don't know how to reliably provoke a visible symptom for the decref problem, though, short of using a debug build and watching for the "negative refcount" message. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-02 01:05 Message: Logged In: YES user_id=21627 Mark, Thanks for your report. It seems Tim is right on all accounts, so there is no need for you to gather further information. I'll try to come up with a scenario where the crash is more reproducible, and check in a fix shortly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 21:24 Message: Logged In: YES user_id=31435 Re-opened this and raised priority. Martin, near the start of PySocket_getnameinfo we've got n = PyArg_ParseTuple(sa, "si|ii", &hostp, &port, &flowinfo, scope_id); and that's surely incorrect (note that we're not passing the address of scope_id there! we're passing its value, and scope_id is uninitialized stack trash). There's more than just that going on here, though. If that's repaired, the PyArg_ParseTuple fails, and so it jumps to the "fail:" label, and sa gets decref'ed there. But sa was obtained from a previous PyArg_ParseTuple call, so should not be decref'ed by this routine (this routine has a borrowed reference to sa). That part almost certainly explains the "negative refcount" msg icep17 sees in a debug build. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 20:15 Message: Logged In: NO I can reproduce it with 2.2b1 on Win9x, and by using that same code. ---------------------------------------------------------------------- Comment By: Mark Rowe (icep17) Date: 2001-11-01 20:05 Message: Logged In: YES user_id=147838 Under Windows ME with Python 2.2b1 I still get the crash. I have asked my friend to check it (Windows 95c Python 2.2b1) and he also gets the crash. I compiled the debug build of Python and running the above code in an interpreter session gives: Adding parser accelerators ... Done. Python 2.2b1 (#25, Oct 21 2001, 13:42:02) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket [8983 refs] >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) C:\Program Files\python22\Python-2.2b1\Objects\tupleobject.c:147 negative ref co unt -572662308 Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple [9020 refs] >>> The Debug build of Python does not crash, but instead reports a negative refrence count, which could be the cause of the crash in the release build. As I am not too familiar with the internals of Python I have trouble understanding what is happening when running the interpreter under the debugger, but if you would like some more detail I will happily provide it. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 06:22 Message: Logged In: YES user_id=21627 I cannot reproduce this in 2.2b1 (it works the same on Win32 as it does on Linux), but I also fail to see what code change may have caused to fail it in 2.2a3. Please update to 2.2b1 and retest; if it still happens, check that you don't have any extra copies of python22.dll. If so, please add a note in this report, and we'll re-open it. If you can, it would help enourmously if you could run the program in a debugger and report the stack trace in case of error, and the file and line number causing the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Fri Nov 2 19:07:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 11:07:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-473060 ] Doc/ doesn't compile in 2.2b1 Message-ID: Bugs item #473060, was opened at 2001-10-20 02:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 Category: Build Group: None >Status: Closed >Resolution: Fixed Priority: 4 Submitted By: Masanori Omote (omote) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Doc/ doesn't compile in 2.2b1 Initial Comment: Documentations in the distribution tar file of 2.2b1 doesn't compile. Doc/ of a snapshot tar ball on 10/19/01 compiles correctly. Materials under Doc/ in both tar balls are the same according to diff -r check. Containing a dot (.) in the name of the top directory of 2.2b1 leads to a LaTeX2HTML error, according to its error message. It wouldn't be a bug because the README file in Doc directory says that a user should "twiddle" Doc/Makefile. But I hope all the documentation can be easily compiled from the next distribution files. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-02 11:07 Message: Logged In: YES user_id=3066 Oops, I made changes to fix this earlier this week, but didn't close the bug report. Here's what I did: - Added a --dvips-safe option to mkhowto to control the LaTeX2HTML variable $DVIPS_SAFE; setting this causes the complaint to go away. - Use --dvips-safe in the Makefile so that we always use it when generating HTML; this is fine with the Python docs since we don't have content that requires dvips. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-29 21:25 Message: Logged In: YES user_id=3066 As noted in the error message, this is because there's a dot in the path to the documentation sources. While I consider this to be primarily a tools problem, it's sufficiently annoying that it deserves some attention. ---------------------------------------------------------------------- Comment By: Masanori Omote (omote) Date: 2001-10-20 09:17 Message: Logged In: YES user_id=232907 Additional info of omote's case: System: RH 7.1 Python: 2.1.1 (2.2b1 not yet installed) LaTeX2HTML: 2K.1beta (1.47) When I compile Doc/ in the extracted tree of 2.2b1 distribution tar file, it fails. It doesn't fail when I move Doc/ directory to any other directory which path doesn't contain a dot. Following is a failure example to compile 'doc'. % pwd /home/masa/ftp/python/Python-2.2b1/Doc % make doc python tools/mkhowto --html --about html/stdabout.dat --address "See About this document... for information on suggesting changes." --up-link ../index.html --up-title "Python Documentation Index" --global-module-index "../modindex.html" --dir html/doc doc/doc.tex +++ TEXINPUTS=/home/masa/ftp/python/Python-2.2b1/Doc/doc:/home/masa/ftp/python/Python-2.2b1/Doc/paper-letter:/home/masa/ftp/python/Python-2.2b1/Doc/texinputs: +++ latex doc +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. The relevant lines from the transcript are: ------------------------------------------------------------------------ +++ latex2html -init_file doc.l2h -dir /home/masa/ftp/python/Python-2.2b1/Doc/html/doc /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex Note: Initialising with file: doc.l2h This is LaTeX2HTML Version 2K.1beta (1.47) by Nikos Drakos, Computer Based Learning Unit, University of Leeds. Revised and extended by: Marcus Hennecke, Ross Moore, Herb Swan and others ...producing markup for HTML version 4.0 Loading /usr/share/latex2html/versions/html4_0.pl *** processing declarations *** Loading /usr/share/latex2html/versions/latin1.pl *** Fatal Error --- but easy to fix *** Cannot have '.' in file-name prefix, else dvips fails on images Change the name from /home/masa/ftp/python/Python-2.2b1/Doc/doc/doc.tex and try again. *** Session transcript and error messages are in /home/masa/ftp/python/Python-2.2b1/Doc/html/doc/doc.how. *** Exited with status 255. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-20 06:31 Message: Logged In: YES user_id=21627 Can you give more details as to how exactly compilation fails? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=473060&group_id=5470 From noreply@sourceforge.net Fri Nov 2 23:36:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 15:36:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-476648 ] socket.getnameinfo crashes interpreter Message-ID: Bugs item #476648, was opened at 2001-10-30 20:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Mark Rowe (icep17) Assigned to: Martin v. Löwis (loewis) Summary: socket.getnameinfo crashes interpreter Initial Comment: When making an invalid call to socket.getnameinfo() it is possible to crash the interpreter. From what I've seen, this occurs under Windows ME with Python 2.2b1 and Windows 98 with Python 2.2a3. It does not occur under Mandrake 8.0 with Python 2.2a3. >>> import socket >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) This results with Python causing an error in MSVCRT.DLL. Under Mandrake 8.0 it causes the expected exception: Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-02 15:36 Message: Logged In: YES user_id=21627 Fixed in socketmodule.c 1.192, test_socket.py 1.21. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-02 09:20 Message: Logged In: YES user_id=31435 Martin, I was wrong about one thing: scope_id isn't uninitialized stack trash, it's forced to 0. So one way to reliably provoke the first problem is simply to pass an argument with a value for scope_id (then the 0 passed for scope_id looks like a NULL pointer to PyArg_ParseTuple). Like, say, >>> socket.getnameinfo(('x', 0, 0, 0), 0) Boom. I don't know how to reliably provoke a visible symptom for the decref problem, though, short of using a debug build and watching for the "negative refcount" message. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-02 01:05 Message: Logged In: YES user_id=21627 Mark, Thanks for your report. It seems Tim is right on all accounts, so there is no need for you to gather further information. I'll try to come up with a scenario where the crash is more reproducible, and check in a fix shortly. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-01 21:24 Message: Logged In: YES user_id=31435 Re-opened this and raised priority. Martin, near the start of PySocket_getnameinfo we've got n = PyArg_ParseTuple(sa, "si|ii", &hostp, &port, &flowinfo, scope_id); and that's surely incorrect (note that we're not passing the address of scope_id there! we're passing its value, and scope_id is uninitialized stack trash). There's more than just that going on here, though. If that's repaired, the PyArg_ParseTuple fails, and so it jumps to the "fail:" label, and sa gets decref'ed there. But sa was obtained from a previous PyArg_ParseTuple call, so should not be decref'ed by this routine (this routine has a borrowed reference to sa). That part almost certainly explains the "negative refcount" msg icep17 sees in a debug build. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 20:15 Message: Logged In: NO I can reproduce it with 2.2b1 on Win9x, and by using that same code. ---------------------------------------------------------------------- Comment By: Mark Rowe (icep17) Date: 2001-11-01 20:05 Message: Logged In: YES user_id=147838 Under Windows ME with Python 2.2b1 I still get the crash. I have asked my friend to check it (Windows 95c Python 2.2b1) and he also gets the crash. I compiled the debug build of Python and running the above code in an interpreter session gives: Adding parser accelerators ... Done. Python 2.2b1 (#25, Oct 21 2001, 13:42:02) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import socket [8983 refs] >>> socket.getnameinfo(socket.inet_aton('127.0.0.1'), 0) C:\Program Files\python22\Python-2.2b1\Objects\tupleobject.c:147 negative ref co unt -572662308 Traceback (most recent call last): File "", line 1, in ? SystemError: new style getargs format but argument is not a tuple [9020 refs] >>> The Debug build of Python does not crash, but instead reports a negative refrence count, which could be the cause of the crash in the release build. As I am not too familiar with the internals of Python I have trouble understanding what is happening when running the interpreter under the debugger, but if you would like some more detail I will happily provide it. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 06:22 Message: Logged In: YES user_id=21627 I cannot reproduce this in 2.2b1 (it works the same on Win32 as it does on Linux), but I also fail to see what code change may have caused to fail it in 2.2a3. Please update to 2.2b1 and retest; if it still happens, check that you don't have any extra copies of python22.dll. If so, please add a note in this report, and we'll re-open it. If you can, it would help enourmously if you could run the program in a debugger and report the stack trace in case of error, and the file and line number causing the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476648&group_id=5470 From noreply@sourceforge.net Sat Nov 3 02:37:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 18:37:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-477700 ] memory leaks in gdbm & curses Message-ID: Bugs item #477700, was opened at 2001-11-02 18:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leaks in gdbm & curses Initial Comment: Purify reports memory leaks in gdbm & curses modules. The attached patch fixes these two small leaks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 From noreply@sourceforge.net Sat Nov 3 06:04:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 02 Nov 2001 22:04:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-477728 ] .*? matches newlines without DOTALL Message-ID: Bugs item #477728, was opened at 2001-11-02 22:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477728&group_id=5470 Category: Regular Expressions Group: Python 2.2 Status: Open Resolution: None Priority: 6 Submitted By: Tim Peters (tim_one) Assigned to: Fredrik Lundh (effbot) Summary: .*? matches newlines without DOTALL Initial Comment: ython 2.2b1+ (#25, Oct 30 2001, 22:36:33) [MSC 32 bit (Intel)] on win32 >> text = "one\ntwo\nthree\n" >> import re >> re.match(r'.*?$', text) SRE_Match object at 0x007AD940> >> _.group(0) # oops! one\ntwo\nthree' >> re.match(r'.*$', text) # more like it >> Since . shouldn't match a newline in the absence of re.DOTALL, .*? shouldn't match any newlines, and then minimal match should have failed (as did the maximal match). Found by Bruce Eckel. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477728&group_id=5470 From noreply@sourceforge.net Sun Nov 4 10:35:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 02:35:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-477963 ] Bad gc performance Message-ID: Bugs item #477963, was opened at 2001-11-04 02:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477963&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Roeland Rengelink (rengelink) Assigned to: Nobody/Anonymous (nobody) Summary: Bad gc performance Initial Comment: The attached program demonstrates bad (maybe O(n**2)) performance of the gc when collecting garbage The program first creates a dictionary d with N 7-character keys referring a unique integer (2N objects) the dictionary is then set to None. The program times the creation of the dictionary (in micro-sec/item) and then the time it takes to execute d=None also expressed in (usec/item). Results are the following: (Python2.2b1, default conf, Linux 2.2.14, Suse 6.2) ~>python timing.py size: 10000, creation: 33.67 (usec/elem), destruction: 1.59 (usec/elem) size: 20000, creation: 33.18 (usec/elem), destruction: 1.67 (usec/elem) size: 50000, creation: 35.48 (usec/elem), destruction: 1.83 (usec/elem) size: 100000, creation: 34.81 (usec/elem), destruction: 2.01 (usec/elem) size: 200000, creation: 34.63 (usec/elem), destruction: 2.50 (usec/elem) size: 400000, creation: 35.12 (usec/elem), destruction: 3.89 (usec/elem) size: 600000, creation: 34.16 (usec/elem), destruction: 6.07 (usec/elem) size: 800000, creation: 34.90 (usec/elem), destruction: 8.10 (usec/elem) size: 1000000, creation: 34.53 (usec/elem), destruction: 11.14 (usec/elem) This test does not exceed main memory (128M) on my machine and CPU utilization remains 95+% throughout. If the test is extended to 2000000 objects on my machine (requires VM) gc leads to an extended period of disk-thrashing (CPU at 1%) requiring many minutes to clean up the garbage. A similar problem was reported by Fred in http://mail.python.org/pipermail/python-list/2001-November/070667.html I'm not sure if this is a bug. My naive expectation is that cleanup should be O(n), but then I don't really kmow what the gc is up to. Let me know if you need more info Roeland ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477963&group_id=5470 From noreply@sourceforge.net Sun Nov 4 14:49:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 06:49:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-477999 ] win32ui.FullPath() fails in PythonWin Message-ID: Bugs item #477999, was opened at 2001-11-04 06:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: win32ui.FullPath() fails in PythonWin Initial Comment: A line in a method of a Class, like the following: self.name = win32ui.FullPath(self.name) works fine when a script is run from the command line. However, it fails when run in PythonWin. From the command line, the full path is set for a filename. In PythonWin, a filename is always shown/returned as being in the root directory of a drive. Using: ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 and the PythonWin that came with this release. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 From noreply@sourceforge.net Sun Nov 4 14:54:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 06:54:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-478001 ] uninit mem read w/signals Message-ID: Bugs item #478001, was opened at 2001-11-04 06:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 Category: Parser/Compiler Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: uninit mem read w/signals Initial Comment: if the signal()/sigaction() fails, uninitialized memory is returned. the attached patch fixes the problem (didn't test signal() path, only the sigaction() path) Neal ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 From noreply@sourceforge.net Sun Nov 4 14:57:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 06:57:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-478002 ] problem in _sre.c Message-ID: Bugs item #478002, was opened at 2001-11-04 06:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478002&group_id=5470 Category: Regular Expressions Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Fredrik Lundh (effbot) Summary: problem in _sre.c Initial Comment: there was a problem reported by purify in sre (sorry forget exact problem) the attached patch corrects the problem. i believe it was a memory leak (doesn't seem right) or perhaps accessing freed memory. not positive the patch fixes the problem in the best way, but purify is happy ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478002&group_id=5470 From noreply@sourceforge.net Sun Nov 4 15:02:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 07:02:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-478003 ] possible memory leaks in err handling Message-ID: Bugs item #478003, was opened at 2001-11-04 07:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478003&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: possible memory leaks in err handling Initial Comment: This patch fixes problems caught during code inspection they seem correct, but I don't think the regression tests hit these paths. I'm not sure how to test them. Files in patch: Modules/getaddrinfo.c Modules/socketmodule.c Python/compile.c ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478003&group_id=5470 From noreply@sourceforge.net Sun Nov 4 15:05:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 07:05:04 -0800 Subject: [Python-bugs-list] [ python-Bugs-478005 ] possible memory leak in posixmodule.c Message-ID: Bugs item #478005, was opened at 2001-11-04 07:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478005&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: possible memory leak in posixmodule.c Initial Comment: when posix_error is called a PyObject is returned, then ignored in the caller. it seems that this might lead to memory leaks. this patch calls Py_DECREF(posix_error()) in all the places that previously ignored the return value. i'm not sure if this is correct or not. i didn't find any problems with the regression tests when running with this patch. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478005&group_id=5470 From noreply@sourceforge.net Sun Nov 4 15:07:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 07:07:17 -0800 Subject: [Python-bugs-list] [ python-Bugs-478006 ] memory leaks in Modules & threads Message-ID: Bugs item #478006, was opened at 2001-11-04 07:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leaks in Modules & threads Initial Comment: the attached patch corrects memory leaks found when running the regression tests. Files: Modules/_hotshot.c Modules/sunaudiodev.c Python/thread_pthread.h ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 From noreply@sourceforge.net Sun Nov 4 15:16:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 07:16:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-477999 ] win32ui.FullPath() fails in PythonWin Message-ID: Bugs item #477999, was opened at 2001-11-04 06:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: win32ui.FullPath() fails in PythonWin Initial Comment: A line in a method of a Class, like the following: self.name = win32ui.FullPath(self.name) works fine when a script is run from the command line. However, it fails when run in PythonWin. From the command line, the full path is set for a filename. In PythonWin, a filename is always shown/returned as being in the root directory of a drive. Using: ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 and the PythonWin that came with this release. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-04 07:16 Message: Logged In: NO This might not be a bug. Apparently PythonWin does not default to the directory of the script that is being run. When I close PythonWin and restart it in the directory of the script that is being run, win32ui.FullPath() works. I don't know if this is intentional behavior or not. It it is not, I suggest that the directory of the script in the "Run Script" dialog of PythonWin be the current/default directory. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 From noreply@sourceforge.net Sun Nov 4 17:19:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 09:19:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-478038 ] urlparse.urlparse semicolon bug Message-ID: Bugs item #478038, was opened at 2001-11-04 09:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: urlparse.urlparse semicolon bug Initial Comment: urlparse,urlparse uses obsolete parsing rules. It expects there to be no more than one semicolon in a URL, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit?x=y It splits the url into parts, one of which is the part after between the semicolon and the question mark. This behavior is based on an obsolete URL spec. Recent specs, including the RFCs referenced in the urlparse documentation allow semicolons in each path, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit/form/spam;eggs=1/splat urlparse.urlparse parses as follows: [jim@c ZServer]$ python2.2 Python 2.2b1 (#1, Oct 22 2001, 17:42:33) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Py$ from urlparse import urlparse Py$ urlparse("http://127.0.0.1:8880/semitest/foo%3Bbar;presentation=edit/form/spam;eggs=1/splat") ('http', '127.0.0.1:8880', '/semitest/foo%3Bbar', 'presentation=edit/form/spam;eggs=1/splat', '', '') Py$ which is incorrect because much of the path is incorrectly included in the obsolete "params" part. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 From noreply@sourceforge.net Sun Nov 4 17:23:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 09:23:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-478002 ] problem in _sre.c Message-ID: Bugs item #478002, was opened at 2001-11-04 06:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478002&group_id=5470 Category: Regular Expressions Group: Python 2.2 >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Fredrik Lundh (effbot) Summary: problem in _sre.c Initial Comment: there was a problem reported by purify in sre (sorry forget exact problem) the attached patch corrects the problem. i believe it was a memory leak (doesn't seem right) or perhaps accessing freed memory. not positive the patch fixes the problem in the best way, but purify is happy ---------------------------------------------------------------------- >Comment By: Fredrik Lundh (effbot) Date: 2001-11-04 09:23 Message: Logged In: YES user_id=38376 probably a false alarm (see python-dev archives for more info). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478002&group_id=5470 From noreply@sourceforge.net Sun Nov 4 18:48:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 10:48:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-477999 ] win32ui.FullPath() fails in PythonWin Message-ID: Bugs item #477999, was opened at 2001-11-04 06:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Mark Hammond (mhammond) Summary: win32ui.FullPath() fails in PythonWin Initial Comment: A line in a method of a Class, like the following: self.name = win32ui.FullPath(self.name) works fine when a script is run from the command line. However, it fails when run in PythonWin. From the command line, the full path is set for a filename. In PythonWin, a filename is always shown/returned as being in the root directory of a drive. Using: ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 and the PythonWin that came with this release. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-04 10:48 Message: Logged In: YES user_id=31435 Reassigned to MarkH. Anonymous, you should really report this to ActiveState. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-04 07:16 Message: Logged In: NO This might not be a bug. Apparently PythonWin does not default to the directory of the script that is being run. When I close PythonWin and restart it in the directory of the script that is being run, win32ui.FullPath() works. I don't know if this is intentional behavior or not. It it is not, I suggest that the directory of the script in the "Run Script" dialog of PythonWin be the current/default directory. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 From noreply@sourceforge.net Sun Nov 4 19:07:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 11:07:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-478005 ] possible memory leak in posixmodule.c Message-ID: Bugs item #478005, was opened at 2001-11-04 07:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478005&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Tim Peters (tim_one) Summary: possible memory leak in posixmodule.c Initial Comment: when posix_error is called a PyObject is returned, then ignored in the caller. it seems that this might lead to memory leaks. this patch calls Py_DECREF(posix_error()) in all the places that previously ignored the return value. i'm not sure if this is correct or not. i didn't find any problems with the regression tests when running with this patch. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-04 11:07 Message: Logged In: YES user_id=31435 Rejected for good cause . Neal, as the comment right before posix_error() says, /* Set a POSIX-specific error from errno, and return NULL */ Doing Py_DECREF(NULL) is almost guaranteed to cause a core dump, so it's hard to believe you tested any of the post- patch changed code (nor did you claim to, but you should have tried to provoke at least one of the error paths patched). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478005&group_id=5470 From noreply@sourceforge.net Sun Nov 4 19:27:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 11:27:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-478003 ] possible memory leaks in err handling Message-ID: Bugs item #478003, was opened at 2001-11-04 07:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478003&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Martin v. Löwis (loewis) Summary: possible memory leaks in err handling Initial Comment: This patch fixes problems caught during code inspection they seem correct, but I don't think the regression tests hit these paths. I'm not sure how to test them. Files in patch: Modules/getaddrinfo.c Modules/socketmodule.c Python/compile.c ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-04 11:27 Message: Logged In: YES user_id=31435 I applied (just) the compile.c part of the patch (thanks!), in Python/compile.c; new revision: 2.228 Reassigned to MvL for the getaddrinfo and socketmodule parts. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478003&group_id=5470 From noreply@sourceforge.net Sun Nov 4 19:45:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 11:45:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-478001 ] uninit mem read w/signals Message-ID: Bugs item #478001, was opened at 2001-11-04 06:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 Category: Parser/Compiler Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Barry Warsaw (bwarsaw) Summary: uninit mem read w/signals Initial Comment: if the signal()/sigaction() fails, uninitialized memory is returned. the attached patch fixes the problem (didn't test signal() path, only the sigaction() path) Neal ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-04 11:45 Message: Logged In: YES user_id=31435 Assigned to Barry. Neal's surely correct that we shouldn't be ignoring errors, but returning NULL isn't right either. Should we force return of SIG_ERR then? That's the only *natural* "error return" value, and there's a backward compatibility problem here since we don't document anything about error returns for the {get,set}sig functions, nor do any of the places we call these check for an error return. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 From noreply@sourceforge.net Sun Nov 4 22:16:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 14:16:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-477963 ] Bad gc performance Message-ID: Bugs item #477963, was opened at 2001-11-04 02:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477963&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Roeland Rengelink (rengelink) >Assigned to: Tim Peters (tim_one) Summary: Bad gc performance Initial Comment: The attached program demonstrates bad (maybe O(n**2)) performance of the gc when collecting garbage The program first creates a dictionary d with N 7-character keys referring a unique integer (2N objects) the dictionary is then set to None. The program times the creation of the dictionary (in micro-sec/item) and then the time it takes to execute d=None also expressed in (usec/item). Results are the following: (Python2.2b1, default conf, Linux 2.2.14, Suse 6.2) ~>python timing.py size: 10000, creation: 33.67 (usec/elem), destruction: 1.59 (usec/elem) size: 20000, creation: 33.18 (usec/elem), destruction: 1.67 (usec/elem) size: 50000, creation: 35.48 (usec/elem), destruction: 1.83 (usec/elem) size: 100000, creation: 34.81 (usec/elem), destruction: 2.01 (usec/elem) size: 200000, creation: 34.63 (usec/elem), destruction: 2.50 (usec/elem) size: 400000, creation: 35.12 (usec/elem), destruction: 3.89 (usec/elem) size: 600000, creation: 34.16 (usec/elem), destruction: 6.07 (usec/elem) size: 800000, creation: 34.90 (usec/elem), destruction: 8.10 (usec/elem) size: 1000000, creation: 34.53 (usec/elem), destruction: 11.14 (usec/elem) This test does not exceed main memory (128M) on my machine and CPU utilization remains 95+% throughout. If the test is extended to 2000000 objects on my machine (requires VM) gc leads to an extended period of disk-thrashing (CPU at 1%) requiring many minutes to clean up the garbage. A similar problem was reported by Fred in http://mail.python.org/pipermail/python-list/2001-November/070667.html I'm not sure if this is a bug. My naive expectation is that cleanup should be O(n), but then I don't really kmow what the gc is up to. Let me know if you need more info Roeland ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-04 14:16 Message: Logged In: YES user_id=31435 My c.l.py response: """ [Roeland Rengelink] > I'm pretty sure this is a problem with the garbage collector, I did some > measurements showing bad (O(n**2)) performance when collecting large > numbers of objects at once (as happens when your dicts are deleted at > the end of your program). I filed a bug report (#477963) on it. See: > > > > for more details I expect you're just measuring your platform's cache performance. Here's the output from running on a Win98SE box w/ 256MB RAM (I'll leave out the creation times because they aren't interesting; note too that time.time() on Windows has poor precision, so the numbers are mostly nonsense for the smallest dicts): size: 10000, destruction: 0.00 (usec/elem) size: 20000, destruction: 2.50 (usec/elem) size: 50000, destruction: 1.20 (usec/elem) size: 100000, destruction: 1.10 (usec/elem) size: 200000, destruction: 1.10 (usec/elem) size: 400000, destruction: 1.25 (usec/elem) size: 600000, destruction: 1.37 (usec/elem) size: 800000, destruction: 1.45 (usec/elem) size: 1000000, destruction: 1.54 (usec/elem) So buy more RAM <0.5 wink>. This is the code that deallocates dicts: for (ep = mp->ma_table; fill > 0; ep++) { if (ep->me_key) { --fill; Py_DECREF(ep->me_key); Py_XDECREF(ep->me_value); } } That is, it's a single scan over the key+value pairs, decrementing the refcounts on each. Note that while a left- to-right scan of the container is cache-friendly, dicts work hard to "randomize" the offsets at which keys are stored, so the memory pointed to *by* ep->me_key (containing your string objects) has no useful relation to the order in which you created your string objects: you're going to get mountains of cache misses during deallocation. It's a mystery why deallocation takes so much longer than allocation when things go bad (look at it: the deallocation code is dead simple). I tracked down one of those long ago in hideous detail, and it turned out to be due to the platform free() trashing like mad trying to coalesce adjacent free'd areas. I suspect every platform has its own tale of woe to relate here. """ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477963&group_id=5470 From noreply@sourceforge.net Sun Nov 4 22:49:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 14:49:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-477700 ] memory leaks in gdbm & curses Message-ID: Bugs item #477700, was opened at 2001-11-02 18:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leaks in gdbm & curses Initial Comment: Purify reports memory leaks in gdbm & curses modules. The attached patch fixes these two small leaks. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-04 14:49 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 From noreply@sourceforge.net Sun Nov 4 23:07:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 15:07:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-478006 ] memory leaks in Modules & threads Message-ID: Bugs item #478006, was opened at 2001-11-04 07:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leaks in Modules & threads Initial Comment: the attached patch corrects memory leaks found when running the regression tests. Files: Modules/_hotshot.c Modules/sunaudiodev.c Python/thread_pthread.h ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-04 15:07 Message: Logged In: YES user_id=31435 Neal, if you do more of these, could you please limit them to one module per patch? Not all the suggested fixes have made sense, and the report gets to be a mess when only part of a patch can be applied. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 From noreply@sourceforge.net Sun Nov 4 23:10:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 04 Nov 2001 15:10:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-478115 ] rfc822.Message.getdate('Date') exception Message-ID: Bugs item #478115, was opened at 2001-11-04 15:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478115&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Piers Lauder (pierslauder) Assigned to: Nobody/Anonymous (nobody) Summary: rfc822.Message.getdate('Date') exception Initial Comment: rfc822.Message.getdate('Date') will raise an IndexError exception if the message header contains a null Date line. An example Python script that reproduces the exception is attached. As far as I know, this affects all versions of rfc822.py. The error is in rfc822.parsedate_tz which assumes that its string argument is non-empty. The first line should probably be: if not data: return None ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478115&group_id=5470 From noreply@sourceforge.net Mon Nov 5 13:39:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 05:39:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-478291 ] 1./4 -> core on IRIX Message-ID: Bugs item #478291, was opened at 2001-11-05 05:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478291&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Dirk Reiners (dirk) Assigned to: Nobody/Anonymous (nobody) Summary: 1./4 -> core on IRIX Initial Comment: We're using Version 2.1.1 on sgi Irix, and the math routines dump core a lot when mixing float and integer numbers. Interestingly enough the effect depends on the actual values of the numbers: tizian: scratch/Python-2.1.1 313 % ./python Python 2.1.1 (#1, Nov 5 2001, 13:34:04) [C] on irix646 Type "copyright", "credits" or "license" for more information. >>> 1./3 0.33333333333333331 >>> 1./5 0.20000000000000001 >>> 1./4 Bus error (core dumped) Or, a little different: tizian: scratch/Python-2.1.1 315 % ./python Python 2.1.1 (#1, Nov 5 2001, 13:34:04) [C] on irix646 Type "copyright", "credits" or "license" for more information. >>> from math import sin >>> 15*sin(8) 14.840373699350726 >>> 17*sin(8) 16.819090192597489 >>> 16*sin(8) Bus error (core dumped) Interestingly enough that only happens with the optimized version, not with the debug version (which makes debugging a little difficult :( ). The stack for the 1./4 case looks like this: > 0 float_div(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/floatobject.c":394, 0x1005561c] 1 binary_op1(0x101a9b48, 0x10149e64, 0x0, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/abstract.c":324, 0x100424a4] 2 binary_op(0x101a9b48, 0x10149e64, 0xc, 0x0, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/abstract.c":373, 0x10042730] 3 PyNumber_Divide(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/abstract.c":545, 0x10042ea4] 4 eval_code2(0x1015b110, 0x0, 0x0, 0x0, 0x1014d949, 0x0, 0x0, 0x0) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/ceval.c":955, 0x1003c10c] 5 PyEval_EvalCode(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/ceval.c":341, 0x10038cd8] 6 run_node(0x10148c90, 0x10149e64, 0x0, 0x0, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":1045, 0x10048618] 7 PyRun_InteractiveOneFlags(0x0, 0x101410f8, 0x0, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":570, 0x10047048] 8 PyRun_InteractiveLoopFlags(0xfb4a520, 0x101410f8, 0x7fff2e30, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":510, 0x10046d88] 9 PyRun_AnyFileExFlags(0xfb4a520, 0x101410f8, 0x0, 0x0, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":473, 0x10046c68] 10 Py_Main(0x1, 0x0, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Modules/main.c":320, 0x10012458] 11 main(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Modules/python.c":10, 0x100120ac] 12 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M4/csu/crt1text.s":177, 0x10012058] It happens here: 390 static PyObject * 391 float_div(PyObject *v, PyObject *w) 392 { 393 double a,b; >* 394 CONVERT_TO_DOUBLE(v, a); 395 CONVERT_TO_DOUBLE(w, b); Just compiling floatobject.c without -O makes it work, but that's not really a solution. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478291&group_id=5470 From noreply@sourceforge.net Mon Nov 5 15:45:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 07:45:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Mon Nov 5 17:24:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 09:24:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-478115 ] rfc822.Message.getdate('Date') exception Message-ID: Bugs item #478115, was opened at 2001-11-04 15:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478115&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Piers Lauder (pierslauder) >Assigned to: Barry Warsaw (bwarsaw) Summary: rfc822.Message.getdate('Date') exception Initial Comment: rfc822.Message.getdate('Date') will raise an IndexError exception if the message header contains a null Date line. An example Python script that reproduces the exception is attached. As far as I know, this affects all versions of rfc822.py. The error is in rfc822.parsedate_tz which assumes that its string argument is non-empty. The first line should probably be: if not data: return None ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478115&group_id=5470 From noreply@sourceforge.net Mon Nov 5 17:29:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 09:29:20 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Mon Nov 5 17:42:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 09:42:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-476616 ] UserDict.copy() error Message-ID: Bugs item #476616, was opened at 2001-10-30 18:57 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476616&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: UserDict.copy() error Initial Comment: The copy() method applied to subclasses of UserDict copies the object, but does not copy the contained dictionary. The dictionary is shared between the original and the copy. The copy method applied to UserDict (not a subclass) works okay. This error goes back to at least 2.0, and still exists in 2.2b1. This little test program demonstrates the error: ---------------------------------------------- from UserDict import UserDict class MyDict (UserDict): pass d = MyDict() d[1] = 11 c = d.copy() c[2] = 22 # this should affect c only, but also affects d print "c:", c print "d:", d --------------------------------------- BTW, subclassing 'dictionary' works fine in 2.2 (of course). For running under 2.0, I worked around this by using copy.deepcopy(). I'm in no hurry for a fix for this. Bob Alexander balexander@rsv.ricoh.com ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 09:42 Message: Logged In: YES user_id=3066 Fixed in UserDict.py revision 1.16 using a different patch. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-30 19:32 Message: Logged In: YES user_id=6380 Hm, you're right. Sigh. This has been there since 1.5.2. I'm not sure how to fix this -- using deepcopy() is too much (and not what happens if self.__class__ is UserDict either). Maybe this: c = copy.copy(self) c.data = self.data.copy() return c ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476616&group_id=5470 From noreply@sourceforge.net Mon Nov 5 17:58:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 09:58:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-478038 ] urlparse.urlparse semicolon bug Message-ID: Bugs item #478038, was opened at 2001-11-04 09:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urlparse.urlparse semicolon bug Initial Comment: urlparse,urlparse uses obsolete parsing rules. It expects there to be no more than one semicolon in a URL, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit?x=y It splits the url into parts, one of which is the part after between the semicolon and the question mark. This behavior is based on an obsolete URL spec. Recent specs, including the RFCs referenced in the urlparse documentation allow semicolons in each path, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit/form/spam;eggs=1/splat urlparse.urlparse parses as follows: [jim@c ZServer]$ python2.2 Python 2.2b1 (#1, Oct 22 2001, 17:42:33) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Py$ from urlparse import urlparse Py$ urlparse("http://127.0.0.1:8880/semitest/foo%3Bbar;presentation=edit/form/spam;eggs=1/splat") ('http', '127.0.0.1:8880', '/semitest/foo%3Bbar', 'presentation=edit/form/spam;eggs=1/splat', '', '') Py$ which is incorrect because much of the path is incorrectly included in the obsolete "params" part. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 From noreply@sourceforge.net Mon Nov 5 18:05:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 10:05:04 -0800 Subject: [Python-bugs-list] [ python-Bugs-450225 ] urljoin fails RFC tests Message-ID: Bugs item #450225, was opened at 2001-08-11 22:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Aaron Swartz (aaronsw) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urljoin fails RFC tests Initial Comment: I've put together a test suite for Python's URLparse module, based on the tests in Appendix C of RFC2396 (the URI RFC). They're available at: http://lists.w3.org/Archives/Public/uri/2001Aug/ 0013.html The major problem seems to be that it treats queries and parameters as special components (not just normal parts of the path), making this related to: http://sourceforge.net/tracker/?group_id=5470& atid=105470&func=detail&aid=210834 ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 10:05 Message: Logged In: YES user_id=3066 This looks like its probably related to #478038; I'll try to tackle them together. Can you attach your tests to the bug report on SF? Thanks! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470 From noreply@sourceforge.net Mon Nov 5 18:30:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 10:30:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-450225 ] urljoin fails RFC tests Message-ID: Bugs item #450225, was opened at 2001-08-11 22:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Aaron Swartz (aaronsw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urljoin fails RFC tests Initial Comment: I've put together a test suite for Python's URLparse module, based on the tests in Appendix C of RFC2396 (the URI RFC). They're available at: http://lists.w3.org/Archives/Public/uri/2001Aug/ 0013.html The major problem seems to be that it treats queries and parameters as special components (not just normal parts of the path), making this related to: http://sourceforge.net/tracker/?group_id=5470& atid=105470&func=detail&aid=210834 ---------------------------------------------------------------------- >Comment By: Aaron Swartz (aaronsw) Date: 2001-11-05 10:30 Message: Logged In: YES user_id=122141 Sure, here they are: import urlparse base = 'http://a/b/c/d;p?q' assert urlparse.urljoin(base, 'g:h') == 'g:h' assert urlparse.urljoin(base, 'g') == 'http://a/b/c/g' assert urlparse.urljoin(base, './g') == 'http://a/b/c/g' assert urlparse.urljoin(base, 'g/') == 'http://a/b/c/g/' assert urlparse.urljoin(base, '/g') == 'http://a/g' assert urlparse.urljoin(base, '//g') == 'http://g' assert urlparse.urljoin(base, '?y') == 'http://a/b/c/?y' assert urlparse.urljoin(base, 'g?y') == 'http://a/b/c/g?y' assert urlparse.urljoin(base, '#s') == 'http://a/b/c/ d;p?q#s' assert urlparse.urljoin(base, 'g#s') == 'http://a/b/c/g#s' assert urlparse.urljoin(base, 'g?y#s') == 'http://a/b/c/ g?y#s' assert urlparse.urljoin(base, ';x') == 'http://a/b/c/;x' assert urlparse.urljoin(base, 'g;x') == 'http://a/b/c/g;x' assert urlparse.urljoin(base, 'g;x?y#s') == 'http://a/b/c/ g;x?y#s' assert urlparse.urljoin(base, '.') == 'http://a/b/c/' assert urlparse.urljoin(base, './') == 'http://a/b/c/' assert urlparse.urljoin(base, '..') == 'http://a/b/' assert urlparse.urljoin(base, '../') == 'http://a/b/' assert urlparse.urljoin(base, '../g') == 'http://a/b/g' assert urlparse.urljoin(base, '../..') == 'http://a/' assert urlparse.urljoin(base, '../../') == 'http://a/' assert urlparse.urljoin(base, '../../g') == 'http://a/g' assert urlparse.urljoin(base, '') == base assert urlparse.urljoin(base, '../../../g') == 'http://a/../g' assert urlparse.urljoin(base, '../../../../g') == 'http://a/../../g' assert urlparse.urljoin(base, '/./g') == 'http://a/./g' assert urlparse.urljoin(base, '/../g') == 'http://a/../g' assert urlparse.urljoin(base, 'g.') == 'http://a/b/c/ g.' assert urlparse.urljoin(base, '.g') == 'http://a/b/c/ .g' assert urlparse.urljoin(base, 'g..') == 'http://a/b/c/ g..' assert urlparse.urljoin(base, '..g') == 'http://a/b/c/ ..g' assert urlparse.urljoin(base, './../g') == 'http://a/b/g' assert urlparse.urljoin(base, './g/.') == 'http://a/b/c/ g/' assert urlparse.urljoin(base, 'g/./h') == 'http://a/b/c/ g/h' assert urlparse.urljoin(base, 'g/../h') == 'http://a/b/c/ h' assert urlparse.urljoin(base, 'g;x=1/./y') == 'http://a/b/c/g;x=1/y' assert urlparse.urljoin(base, 'g;x=1/../y') == 'http://a/b/ c/y' assert urlparse.urljoin(base, 'g?y/./x') == 'http://a/b/c/g?y/./x' assert urlparse.urljoin(base, 'g?y/../x') == 'http://a/b/c/g?y/../x' assert urlparse.urljoin(base, 'g#s/./x') == 'http://a/b/ c/g#s/./x' assert urlparse.urljoin(base, 'g#s/../x') == 'http://a/b/ c/g#s/../x' ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 10:05 Message: Logged In: YES user_id=3066 This looks like its probably related to #478038; I'll try to tackle them together. Can you attach your tests to the bug report on SF? Thanks! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470 From noreply@sourceforge.net Mon Nov 5 18:34:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 10:34:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-450225 ] urljoin fails RFC tests Message-ID: Bugs item #450225, was opened at 2001-08-11 22:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Aaron Swartz (aaronsw) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urljoin fails RFC tests Initial Comment: I've put together a test suite for Python's URLparse module, based on the tests in Appendix C of RFC2396 (the URI RFC). They're available at: http://lists.w3.org/Archives/Public/uri/2001Aug/ 0013.html The major problem seems to be that it treats queries and parameters as special components (not just normal parts of the path), making this related to: http://sourceforge.net/tracker/?group_id=5470& atid=105470&func=detail&aid=210834 ---------------------------------------------------------------------- >Comment By: Aaron Swartz (aaronsw) Date: 2001-11-05 10:34 Message: Logged In: YES user_id=122141 Oops, meant to attach it... ---------------------------------------------------------------------- Comment By: Aaron Swartz (aaronsw) Date: 2001-11-05 10:30 Message: Logged In: YES user_id=122141 Sure, here they are: import urlparse base = 'http://a/b/c/d;p?q' assert urlparse.urljoin(base, 'g:h') == 'g:h' assert urlparse.urljoin(base, 'g') == 'http://a/b/c/g' assert urlparse.urljoin(base, './g') == 'http://a/b/c/g' assert urlparse.urljoin(base, 'g/') == 'http://a/b/c/g/' assert urlparse.urljoin(base, '/g') == 'http://a/g' assert urlparse.urljoin(base, '//g') == 'http://g' assert urlparse.urljoin(base, '?y') == 'http://a/b/c/?y' assert urlparse.urljoin(base, 'g?y') == 'http://a/b/c/g?y' assert urlparse.urljoin(base, '#s') == 'http://a/b/c/ d;p?q#s' assert urlparse.urljoin(base, 'g#s') == 'http://a/b/c/g#s' assert urlparse.urljoin(base, 'g?y#s') == 'http://a/b/c/ g?y#s' assert urlparse.urljoin(base, ';x') == 'http://a/b/c/;x' assert urlparse.urljoin(base, 'g;x') == 'http://a/b/c/g;x' assert urlparse.urljoin(base, 'g;x?y#s') == 'http://a/b/c/ g;x?y#s' assert urlparse.urljoin(base, '.') == 'http://a/b/c/' assert urlparse.urljoin(base, './') == 'http://a/b/c/' assert urlparse.urljoin(base, '..') == 'http://a/b/' assert urlparse.urljoin(base, '../') == 'http://a/b/' assert urlparse.urljoin(base, '../g') == 'http://a/b/g' assert urlparse.urljoin(base, '../..') == 'http://a/' assert urlparse.urljoin(base, '../../') == 'http://a/' assert urlparse.urljoin(base, '../../g') == 'http://a/g' assert urlparse.urljoin(base, '') == base assert urlparse.urljoin(base, '../../../g') == 'http://a/../g' assert urlparse.urljoin(base, '../../../../g') == 'http://a/../../g' assert urlparse.urljoin(base, '/./g') == 'http://a/./g' assert urlparse.urljoin(base, '/../g') == 'http://a/../g' assert urlparse.urljoin(base, 'g.') == 'http://a/b/c/ g.' assert urlparse.urljoin(base, '.g') == 'http://a/b/c/ .g' assert urlparse.urljoin(base, 'g..') == 'http://a/b/c/ g..' assert urlparse.urljoin(base, '..g') == 'http://a/b/c/ ..g' assert urlparse.urljoin(base, './../g') == 'http://a/b/g' assert urlparse.urljoin(base, './g/.') == 'http://a/b/c/ g/' assert urlparse.urljoin(base, 'g/./h') == 'http://a/b/c/ g/h' assert urlparse.urljoin(base, 'g/../h') == 'http://a/b/c/ h' assert urlparse.urljoin(base, 'g;x=1/./y') == 'http://a/b/c/g;x=1/y' assert urlparse.urljoin(base, 'g;x=1/../y') == 'http://a/b/ c/y' assert urlparse.urljoin(base, 'g?y/./x') == 'http://a/b/c/g?y/./x' assert urlparse.urljoin(base, 'g?y/../x') == 'http://a/b/c/g?y/../x' assert urlparse.urljoin(base, 'g#s/./x') == 'http://a/b/ c/g#s/./x' assert urlparse.urljoin(base, 'g#s/../x') == 'http://a/b/ c/g#s/../x' ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 10:05 Message: Logged In: YES user_id=3066 This looks like its probably related to #478038; I'll try to tackle them together. Can you attach your tests to the bug report on SF? Thanks! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=450225&group_id=5470 From noreply@sourceforge.net Mon Nov 5 19:23:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 11:23:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-478291 ] 1./4 -> core on IRIX Message-ID: Bugs item #478291, was opened at 2001-11-05 05:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478291&group_id=5470 Category: Python Interpreter Core >Group: Platform-specific >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Dirk Reiners (dirk) >Assigned to: Tim Peters (tim_one) Summary: 1./4 -> core on IRIX Initial Comment: We're using Version 2.1.1 on sgi Irix, and the math routines dump core a lot when mixing float and integer numbers. Interestingly enough the effect depends on the actual values of the numbers: tizian: scratch/Python-2.1.1 313 % ./python Python 2.1.1 (#1, Nov 5 2001, 13:34:04) [C] on irix646 Type "copyright", "credits" or "license" for more information. >>> 1./3 0.33333333333333331 >>> 1./5 0.20000000000000001 >>> 1./4 Bus error (core dumped) Or, a little different: tizian: scratch/Python-2.1.1 315 % ./python Python 2.1.1 (#1, Nov 5 2001, 13:34:04) [C] on irix646 Type "copyright", "credits" or "license" for more information. >>> from math import sin >>> 15*sin(8) 14.840373699350726 >>> 17*sin(8) 16.819090192597489 >>> 16*sin(8) Bus error (core dumped) Interestingly enough that only happens with the optimized version, not with the debug version (which makes debugging a little difficult :( ). The stack for the 1./4 case looks like this: > 0 float_div(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/floatobject.c":394, 0x1005561c] 1 binary_op1(0x101a9b48, 0x10149e64, 0x0, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/abstract.c":324, 0x100424a4] 2 binary_op(0x101a9b48, 0x10149e64, 0xc, 0x0, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/abstract.c":373, 0x10042730] 3 PyNumber_Divide(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Objects/abstract.c":545, 0x10042ea4] 4 eval_code2(0x1015b110, 0x0, 0x0, 0x0, 0x1014d949, 0x0, 0x0, 0x0) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/ceval.c":955, 0x1003c10c] 5 PyEval_EvalCode(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/ceval.c":341, 0x10038cd8] 6 run_node(0x10148c90, 0x10149e64, 0x0, 0x0, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":1045, 0x10048618] 7 PyRun_InteractiveOneFlags(0x0, 0x101410f8, 0x0, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":570, 0x10047048] 8 PyRun_InteractiveLoopFlags(0xfb4a520, 0x101410f8, 0x7fff2e30, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":510, 0x10046d88] 9 PyRun_AnyFileExFlags(0xfb4a520, 0x101410f8, 0x0, 0x0, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Python/pythonrun.c":473, 0x10046c68] 10 Py_Main(0x1, 0x0, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Modules/main.c":320, 0x10012458] 11 main(0x101a9b48, 0x10149e64, 0xc, 0x101416dc, 0x100550d0, 0x100f4460, 0x100f44d0, 0x1) ["/igd/a4/scratch/reiners/Python-2.1.1/Modules/python.c":10, 0x100120ac] 12 __start() ["/xlv55/kudzu-apr12/work/irix/lib/libc/libc_n32_M4/csu/crt1text.s":177, 0x10012058] It happens here: 390 static PyObject * 391 float_div(PyObject *v, PyObject *w) 392 { 393 double a,b; >* 394 CONVERT_TO_DOUBLE(v, a); 395 CONVERT_TO_DOUBLE(w, b); Just compiling floatobject.c without -O makes it work, but that's not really a solution. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-05 11:23 Message: Logged In: YES user_id=31435 Closed as Duplicate; see bug 435026 for details. It's an optimization bug in the SGI compiler. You should ask SGI for a compiler fix. Over the years we've had so many -O problems on SGI that Python's main README file has long suggested turning off optimization on SGI in case of any strange behavior on that platform. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478291&group_id=5470 From noreply@sourceforge.net Mon Nov 5 20:12:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 12:12:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-478421 ] cPickle bug when using imp.load_source() Message-ID: Bugs item #478421, was opened at 2001-11-05 12:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478421&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Andrew R Gross (sincere2001) Assigned to: Nobody/Anonymous (nobody) Summary: cPickle bug when using imp.load_source() Initial Comment: When a Python 2.1.1 program dynamically creates a module using the 'imp.load_source()' function, and the created module tries to 'import cPickle', the interpreter dies with a SystemError exception: SystemError: _PyImport_FixupExtension: module import_cPickle.cPickle not loaded This does *not* occur when the find/load_module() functions are used. The bug also appears to have been introduced somewhere in 2.x, as 1.5.x works fine with load_source(). Attached is a full explanation of the behavior and some small sample programs to illustrate the bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478421&group_id=5470 From noreply@sourceforge.net Mon Nov 5 20:19:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 12:19:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-478425 ] Change in os.path.join (ntpath.py) Message-ID: Bugs item #478425, was opened at 2001-11-05 12:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478425&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Greg Chapman (glchapman) Assigned to: Nobody/Anonymous (nobody) Summary: Change in os.path.join (ntpath.py) Initial Comment: I had been using this: os.path.join(dirname, "") as a convenient way of getting a dirname with a path separator appended if it didn't have one already. Under Python 2.2b1 in Windows, this no longer works. Looking at the documentation (specifically, the qualifier "unless path is empty"), it appears that it never should have worked. So, I guess that's not a bug. However, in figuring this out, I noticed that the join in ntpath.py is now significantly different from the join in the other path modules. And those other modules will still allow a blank string as the last parameter to produce a pathstring with a terminating path separator, so I guess they need to be updated. Anyway, the different join behavior clearly needs to be synchronized between platforms. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478425&group_id=5470 From noreply@sourceforge.net Mon Nov 5 20:24:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 12:24:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-478428 ] dbm build fails Message-ID: Bugs item #478428, was opened at 2001-11-05 12:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478428&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: John Buell (dadaist) Assigned to: Jack Jansen (jackjansen) Summary: dbm build fails Initial Comment: Trying a build with cvs source downloaded this morning. I've been sure to do the configure --with-suffix=_ on case-insensitive Mac OS X (10.1), but I'm having some problem with the 'dbm' module. Here's a copy of the messages: building 'dbm' extension /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c: In function `dbm_subscript': /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c:104: warning: implicit declaration of function `dbm_error' /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c:105: warning: implicit declaration of function `dbm_clearerr' cc -g -O3 -Wall -Wstrict-prototypes -no-cpp-precomp -I. -I/Users/jbuell/Documents/Python/python/dist/src/./Include -I/Users/jbuell/Documents/Python/python/dist/src/./Mac/Include -I/usr/local/include -IInclude/ -c /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c -o build/temp.darwin-1.4-Power Macintosh-2.2/dbmmodule.o cc -bundle -flat_namespace -undefined suppress build/temp.darwin-1.4-Power Macintosh-2.2/dbmmodule.o -L/usr/local/lib -o build/lib.darwin-1.4-Power Macintosh-2.2/dbm.so building 'gdbm' extension dyld: ./python_ Undefined symbols: __ashldi3 make: *** [sharedmods] Error 67 Any suggestions? Anything I can change in my settings to get past this? -John ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478428&group_id=5470 From noreply@sourceforge.net Mon Nov 5 21:31:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 13:31:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-478425 ] Change in os.path.join (ntpath.py) Message-ID: Bugs item #478425, was opened at 2001-11-05 12:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478425&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Greg Chapman (glchapman) >Assigned to: Tim Peters (tim_one) Summary: Change in os.path.join (ntpath.py) Initial Comment: I had been using this: os.path.join(dirname, "") as a convenient way of getting a dirname with a path separator appended if it didn't have one already. Under Python 2.2b1 in Windows, this no longer works. Looking at the documentation (specifically, the qualifier "unless path is empty"), it appears that it never should have worked. So, I guess that's not a bug. However, in figuring this out, I noticed that the join in ntpath.py is now significantly different from the join in the other path modules. And those other modules will still allow a blank string as the last parameter to produce a pathstring with a terminating path separator, so I guess they need to be updated. Anyway, the different join behavior clearly needs to be synchronized between platforms. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-05 13:31 Message: Logged In: YES user_id=31435 Ya, ntpath.join() became a friggin' mess after fixing a slew of bugs involving Windows "drive letters" (which other platforms can blissfully ignore). I consider the new behavior in this case more of a bug than not, because ntpath.split('a\') produces ('a', ''), and it's best if join() acts like split()'s inverse (when possible). So I restored the old behavior in this case: Lib/ntpath.py; new revision: 1.44 Lib/test/test_ntpath.py; new revision: 1.13 Cases to note: ntpath.join('') -> '' ntpath.join('', '', '', '', '') -> '' ntpath.join('a') -> 'a' ntpath.join('', 'a') -> 'a ntpath.join('', '', '', '', 'a') -> 'a' ntpath.join('a', '') -> 'a\' ntpath.join('a', '', '', '', '') -> 'a\' ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478425&group_id=5470 From noreply@sourceforge.net Mon Nov 5 21:35:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 13:35:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-478038 ] urlparse.urlparse semicolon bug Message-ID: Bugs item #478038, was opened at 2001-11-04 09:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urlparse.urlparse semicolon bug Initial Comment: urlparse,urlparse uses obsolete parsing rules. It expects there to be no more than one semicolon in a URL, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit?x=y It splits the url into parts, one of which is the part after between the semicolon and the question mark. This behavior is based on an obsolete URL spec. Recent specs, including the RFCs referenced in the urlparse documentation allow semicolons in each path, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit/form/spam;eggs=1/splat urlparse.urlparse parses as follows: [jim@c ZServer]$ python2.2 Python 2.2b1 (#1, Oct 22 2001, 17:42:33) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Py$ from urlparse import urlparse Py$ urlparse("http://127.0.0.1:8880/semitest/foo%3Bbar;presentation=edit/form/spam;eggs=1/splat") ('http', '127.0.0.1:8880', '/semitest/foo%3Bbar', 'presentation=edit/form/spam;eggs=1/splat', '', '') Py$ which is incorrect because much of the path is incorrectly included in the obsolete "params" part. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 13:35 Message: Logged In: YES user_id=3066 Here's my proposal for a fix: For the existing urlparse() function, return something in the params field of the result tuple only if it appears on the last path segment. This makes it an empty string for your example, but for URLs which conform to the simpler version of the specifications the API was designed for continue to give the expected behavior. To support the current RFC 2396 syntax, a new function is needed which returns a 5-tuple (the current 6-tuple less the params field). A second new function can be provided which splits the path component into a sequence of pairs, which each pair is (namepart, params). Does this seem acceptable? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 From noreply@sourceforge.net Mon Nov 5 22:47:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 14:47:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-478038 ] urlparse.urlparse semicolon bug Message-ID: Bugs item #478038, was opened at 2001-11-04 09:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urlparse.urlparse semicolon bug Initial Comment: urlparse,urlparse uses obsolete parsing rules. It expects there to be no more than one semicolon in a URL, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit?x=y It splits the url into parts, one of which is the part after between the semicolon and the question mark. This behavior is based on an obsolete URL spec. Recent specs, including the RFCs referenced in the urlparse documentation allow semicolons in each path, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit/form/spam;eggs=1/splat urlparse.urlparse parses as follows: [jim@c ZServer]$ python2.2 Python 2.2b1 (#1, Oct 22 2001, 17:42:33) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Py$ from urlparse import urlparse Py$ urlparse("http://127.0.0.1:8880/semitest/foo%3Bbar;presentation=edit/form/spam;eggs=1/splat") ('http', '127.0.0.1:8880', '/semitest/foo%3Bbar', 'presentation=edit/form/spam;eggs=1/splat', '', '') Py$ which is incorrect because much of the path is incorrectly included in the obsolete "params" part. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 14:47 Message: Logged In: YES user_id=3066 I've attached a patch that makes the change described in my previous comment and cleans up the code a little. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 13:35 Message: Logged In: YES user_id=3066 Here's my proposal for a fix: For the existing urlparse() function, return something in the params field of the result tuple only if it appears on the last path segment. This makes it an empty string for your example, but for URLs which conform to the simpler version of the specifications the API was designed for continue to give the expected behavior. To support the current RFC 2396 syntax, a new function is needed which returns a 5-tuple (the current 6-tuple less the params field). A second new function can be provided which splits the path component into a sequence of pairs, which each pair is (namepart, params). Does this seem acceptable? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 From noreply@sourceforge.net Mon Nov 5 22:59:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 14:59:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-478486 ] Example program locks up IDLE Message-ID: Bugs item #478486, was opened at 2001-11-05 14:59 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478486&group_id=5470 Category: IDLE Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: Example program locks up IDLE Initial Comment: Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Windows NT 4.x Run the example threads code from the book _Core Python Programming_ by Wesley Chu. IDLE locks up and has to be killed from the Task Manager. Call main() in the module defined here: import threading from time import sleep, time, ctime loops = [4,2] def loop(nloop, nsec): print 'start ', nloop, ' at: ', ctime(time()) sleep(nsec) print 'loop ', nloop, ' done at ', ctime(time()) def main(): print 'starting threads' threads = [] nloops = range(len(loops)) for i in nloops: t = threading.Thread(target=loop,args=(i,loops[i])) threads.append(t) for i in nloops: threads[i].start() for i in nloops: threads[i].join() print 'all done at ', ctime(time()) if __name__ == '__main__': main() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478486&group_id=5470 From noreply@sourceforge.net Mon Nov 5 23:21:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 15:21:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-478486 ] Example program locks up IDLE Message-ID: Bugs item #478486, was opened at 2001-11-05 14:59 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478486&group_id=5470 Category: IDLE >Group: None >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: Example program locks up IDLE Initial Comment: Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Windows NT 4.x Run the example threads code from the book _Core Python Programming_ by Wesley Chu. IDLE locks up and has to be killed from the Task Manager. Call main() in the module defined here: import threading from time import sleep, time, ctime loops = [4,2] def loop(nloop, nsec): print 'start ', nloop, ' at: ', ctime(time()) sleep(nsec) print 'loop ', nloop, ' done at ', ctime(time()) def main(): print 'starting threads' threads = [] nloops = range(len(loops)) for i in nloops: t = threading.Thread(target=loop,args=(i,loops[i])) threads.append(t) for i in nloops: threads[i].start() for i in nloops: threads[i].join() print 'all done at ', ctime(time()) if __name__ == '__main__': main() ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-05 15:21 Message: Logged In: YES user_id=31435 >From PEP 42: "IDLE has deep problems running threaded programs. Re- architect." Unless you know exactly what you're doing, running a threaded program under IDLE is almost guaranteed to lock up (several GUI systems have this problem, BTW). Until someone rewrites IDLE, you'll have to run thread examples from a command-line shell. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478486&group_id=5470 From noreply@sourceforge.net Tue Nov 6 03:16:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 19:16:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Nobody/Anonymous (nobody) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Tue Nov 6 03:27:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 19:27:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-478536 ] TypeError from WeakKeyDictionary.has_key Message-ID: Bugs item #478536, was opened at 2001-11-05 19:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478536&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Nobody/Anonymous (nobody) Summary: TypeError from WeakKeyDictionary.has_key Initial Comment: TypeError from WeakKeyDictionary.has_key The has_key method on WeakKeyDictionary gives a TypeError if the key is not weakly referencable. The result I expected was 0. I can put an exception catcher around the call, but I think the right place for that catcher is in the has_key method. Sverker Nilsson Example: Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import weakref >>> ref = weakref.WeakKeyDictionary() >>> ref.has_key('asdf') Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/weakref.py", line 182, in has_key return self.data.has_key(ref(key)) TypeError: 'str' objects are not weakly referencable >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478536&group_id=5470 From noreply@sourceforge.net Tue Nov 6 07:17:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 05 Nov 2001 23:17:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Nobody/Anonymous (nobody) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Tue Nov 6 12:42:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 04:42:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific Status: Closed Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-06 04:42 Message: Logged In: NO gcc --version is now 3.0.1 The problem however is still there: building '_socket' extension gcc -g -O2 -Wall -Wstrict-prototypes -fpic -I. -I/utmnt/ut/si/dv0216/Python-2.1.1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.1/socketmodule.o In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:145: /usr/include/sys/socket.h:439: parse error before "sendfile" /usr/include/sys/socket.h:439: parse error before "bsize_t" /usr/include/sys/socket.h:441: parse error before "sendpath" /usr/include/sys/socket.h:441: parse error before "bsize_t" /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_accept': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:795: warning: passing arg 3 of `accept' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getsockopt': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:978: warning: passing arg 5 of `getsockopt' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:992: warning: passing arg 5 of `getsockopt' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getsockname': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1188: warning: passing arg 3 of `getsockname' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getpeername': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1218: warning: passing arg 3 of `getpeername' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_recvfrom': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1376: warning: passing arg 6 of `recvfrom' from incompatible pointer type WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 Here are the offending lines from socket.h extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); However, my medium c skills tell me that the lines are ok. I cannot offer an explanation why gcc fails here. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 08:57 Message: Logged In: YES user_id=21627 gcc 2.95 does not support HP-UX 11; this appears to be an instance of that problem. The __va__list problem is not at all Python-specific; it occurs in many packages (just search Google for "HP-UX __va__list"). With such problems, there is no point into looking further into the problem; just drop the compiler. If you want to port Python to HP-UX and gcc 2.95, you are on your own. Notice that gcc 3.0.1 *does* support HP-UX 11; I recommend to upgrade. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 07:41 Message: Logged In: NO gcc version is 2.95.3 HP-UX hp18 B.11.11 U 9000/785 2011589119 both Python 2.1.1 and 2.2b1 do not compile ootb here. Error messages during compile are building '_socket' extension gcc -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/utmnt/ut/si/dv0216/Python-2.2b1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.2/socketmodule.o In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:42, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: `__va__list' redefined /usr/include/sys/stdsyms.h:422: warning: this is the location of the previous definition In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:47, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/string.h:26: warning: `__va__list' redefined /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: this is the location of the previous definition In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:152: /usr/include/sys/socket.h:439: parse error before `sendfile' /usr/include/sys/socket.h:439: parse error before `bsize_t' /usr/include/sys/socket.h:440: warning: data definition has no type or storage class /usr/include/sys/socket.h:441: parse error before `sendpath' /usr/include/sys/socket.h:441: parse error before `bsize_t' /usr/include/sys/socket.h:442: warning: data definition has no type or storage class /usr/include/sys/socket.h:456: parse error before `__sendfile64' /usr/include/sys/socket.h:456: parse error before `bsize_t' /usr/include/sys/socket.h:456: warning: data definition has no type or storage class /usr/include/sys/socket.h:457: parse error before `__sendpath64' /usr/include/sys/socket.h:457: parse error before `bsize_t' /usr/include/sys/socket.h:457: warning: data definition has no type or storage class /usr/include/sys/socket.h:459: parse error before `sendfile' /usr/include/sys/socket.h:459: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendfile': /usr/include/sys/socket.h:459: parse error before `bsize_t' /usr/include/sys/socket.h: At top level: /usr/include/sys/socket.h:460: parse error before `sendpath' /usr/include/sys/socket.h:460: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendpath': /usr/include/sys/socket.h:460: parse error before `bsize_t' In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:238: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: At top level: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:203: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:212: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `freeaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:219: warning: implicit declaration of function `free' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `str_isnumber': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:231: warning: subscript has type `char' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `getaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:345: warning: implicit declaration of function `atoi' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:396: warning: implicit declaration of function `malloc' WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 I now thought that since the errors happen in a system header that compiles with cc and not gcc, why not copy to "socket.h" and delete the two offending lines. However, as you see, is still included (why ?). Another option is using cc just for this library, but I'm not sure, that the object files are compatible. Using cc alone creates a lot of non-working modules, although it is in (extended) ANSI mode. I also tried the HP Porting centre (http://hpux.asknet.de) but the Python 2.1 package there -just like mine- does crash on importing socket (too with 2.2b1) hp18: Python-2.2b1 % ./python Python 2.2b1 (#4, Nov 1 2001, 16:36:54) [C] on hp-uxB Type "help", "copyright", "credits" or "license" for more information. >>> import socket Traceback (most recent call last): File "", line 1, in ? File "/utmnt/ut/si/dv0216/Python-2.2b1/Lib/socket.py", line 41, in ? from _socket import * ImportError: No module named _socket >>> ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-30 00:56 Message: Logged In: YES user_id=21627 It would be helpful if you would provide all details in a report also. When you say "it crashes", did you really mean "it crashes" (i.e. with a core dump, machine reboot, or the like)? If the compiler merely rejected the code, it would be good to see what the error message was. Please also report version numbers: HP-UX version, gcc version, etc. Are you sure you are using a gcc built for your OS version? If the gcc doesn't get prototypes right, this often comes from having fixincluded header files of a different OS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 04:59 Message: Logged In: YES user_id=6380 Please try again with Python 2.2b1 (http://python.org/2.2/). We've done a lot of work and we've got at least one report that it now works out of the box on HP-UX 11. I'd like to hear if that solves your problems (and if 2.2b1 builds with either compiler). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Tue Nov 6 16:08:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 08:08:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-478536 ] TypeError from WeakKeyDictionary.has_key Message-ID: Bugs item #478536, was opened at 2001-11-05 19:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478536&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Sverker Nilsson (svenil) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: TypeError from WeakKeyDictionary.has_key Initial Comment: TypeError from WeakKeyDictionary.has_key The has_key method on WeakKeyDictionary gives a TypeError if the key is not weakly referencable. The result I expected was 0. I can put an exception catcher around the call, but I think the right place for that catcher is in the has_key method. Sverker Nilsson Example: Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import weakref >>> ref = weakref.WeakKeyDictionary() >>> ref.has_key('asdf') Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/weakref.py", line 182, in has_key return self.data.has_key(ref(key)) TypeError: 'str' objects are not weakly referencable >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478536&group_id=5470 From noreply@sourceforge.net Tue Nov 6 16:08:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 08:08:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Sverker Nilsson (svenil) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Tue Nov 6 16:33:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 08:33:38 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Pending >Resolution: Works For Me Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:33 Message: Logged In: YES user_id=3066 I cannot reproduce this using either code snippet. Note that the second example should (and does for me) raise TypeError when calling add() -- you left in a "self" that appears to be unintended. Are you using 2.2b1 or CVS python? ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Tue Nov 6 16:39:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 08:39:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-478536 ] TypeError from WeakKeyDictionary.has_key Message-ID: Bugs item #478536, was opened at 2001-11-05 19:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478536&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: TypeError from WeakKeyDictionary.has_key Initial Comment: TypeError from WeakKeyDictionary.has_key The has_key method on WeakKeyDictionary gives a TypeError if the key is not weakly referencable. The result I expected was 0. I can put an exception catcher around the call, but I think the right place for that catcher is in the has_key method. Sverker Nilsson Example: Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import weakref >>> ref = weakref.WeakKeyDictionary() >>> ref.has_key('asdf') Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.2/weakref.py", line 182, in has_key return self.data.has_key(ref(key)) TypeError: 'str' objects are not weakly referencable >>> ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:39 Message: Logged In: YES user_id=3066 Agreed. Fixed in Lib/weakref.py revision 1.15. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478536&group_id=5470 From noreply@sourceforge.net Tue Nov 6 16:43:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 08:43:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-478764 ] does not install on Windows XP Message-ID: Bugs item #478764, was opened at 2001-11-06 08:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 Category: Installation Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: does not install on Windows XP Initial Comment: When executing the BeOpen-Python-2.0.exe installation file, I get the following error from the installer: Error: Could not write file, your disk may be full. The drive I executed the file on has 143GB free. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 From noreply@sourceforge.net Tue Nov 6 16:53:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 08:53:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-478768 ] type attribute hides member of same name Message-ID: Bugs item #478768, was opened at 2001-11-06 08:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael McLay (mclay) Assigned to: Nobody/Anonymous (nobody) Summary: type attribute hides member of same name Initial Comment: Allowing type attributes to replace member descriptors of the same name removes access to all instance data of the same name throughout the application. The error generated is not obvious and may not occur near the source of the error. A patch to prevent this is attached. The use of __slots__ to define attributes allowed in an instance of a type eliminates the __dict__ in instance. Instead the names of the instance attributes are declared in the __slots__ attribute of the class definition. The names defined in __slots__ become member descriptor in the type dict. This change has useful advantages, but there is one side effect that makes the use of __slots__ different from the semantics of the old style dynamic classes. It is not possible to have a type attribute and an instance attribute with the same name. This eliminates the idiom of using a class attribute as a default and then overriding the default value by creating an instance attribute of the same name. This is no longer possible because the declaration of a type attribute name will replace the definition of the member name in the type dict. Once this occurs it impossible to access any instances of the attribute of the same name as a type attribute because the set and get methods for accessing the data are held in the member descriptor. The AttributeError raised in the following example illustrates the problem with using the names declared in __slots__ when naming a type attribute. >>> class B(object): ... a = 3 ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> The member descriptor 'a' defined in __slots__ is inaccessible because the type dict has overwritten the member descriptor with the type attribute of the same name. A more descriptive error message can be generated by checking that no __slots__ names are in the type dict when the __slots__ are being created by type_new(). An instance member defined by __slots__ can also be hidden by a type attribute following the completion of the definition of the class by making an assignment to the type with the same name as the instance member. In the following example the "B.a = "hiding b.a" replaces the reference to the member descriptor for instance member 'a' in the type dict. This eliminates access to all instance members of that name throughout the application. >>> class B(object): ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 >>> B.a = "hiding b.a" >>> b.a 'hiding b.a' >>> b.a = 5 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 From noreply@sourceforge.net Tue Nov 6 17:37:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 09:37:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-478764 ] does not install on Windows XP Message-ID: Bugs item #478764, was opened at 2001-11-06 08:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 Category: Installation Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: does not install on Windows XP Initial Comment: When executing the BeOpen-Python-2.0.exe installation file, I get the following error from the installer: Error: Could not write file, your disk may be full. The drive I executed the file on has 143GB free. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-06 09:37 Message: Logged In: YES user_id=31435 Were you logged in to an Administrator account? If not, try that. If so, this may be hopeless: the 2.0 installer was built with a very old version of Wise, written before even Win98 existed, and is a 16-bit program that started having problems on Win2000 already. Ditto the 2.1 installer. The 2.2 installer is built with a recent version of Wise, and is a 32-bit program. We don't have XP, though, so can't say for sure. You may also have better luck using ActiveState's distribution (which uses different installation technology). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 From noreply@sourceforge.net Tue Nov 6 17:43:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 09:43:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-478768 ] type attribute hides member of same name Message-ID: Bugs item #478768, was opened at 2001-11-06 08:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 >Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael McLay (mclay) >Assigned to: Tim Peters (tim_one) Summary: type attribute hides member of same name Initial Comment: Allowing type attributes to replace member descriptors of the same name removes access to all instance data of the same name throughout the application. The error generated is not obvious and may not occur near the source of the error. A patch to prevent this is attached. The use of __slots__ to define attributes allowed in an instance of a type eliminates the __dict__ in instance. Instead the names of the instance attributes are declared in the __slots__ attribute of the class definition. The names defined in __slots__ become member descriptor in the type dict. This change has useful advantages, but there is one side effect that makes the use of __slots__ different from the semantics of the old style dynamic classes. It is not possible to have a type attribute and an instance attribute with the same name. This eliminates the idiom of using a class attribute as a default and then overriding the default value by creating an instance attribute of the same name. This is no longer possible because the declaration of a type attribute name will replace the definition of the member name in the type dict. Once this occurs it impossible to access any instances of the attribute of the same name as a type attribute because the set and get methods for accessing the data are held in the member descriptor. The AttributeError raised in the following example illustrates the problem with using the names declared in __slots__ when naming a type attribute. >>> class B(object): ... a = 3 ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> The member descriptor 'a' defined in __slots__ is inaccessible because the type dict has overwritten the member descriptor with the type attribute of the same name. A more descriptive error message can be generated by checking that no __slots__ names are in the type dict when the __slots__ are being created by type_new(). An instance member defined by __slots__ can also be hidden by a type attribute following the completion of the definition of the class by making an assignment to the type with the same name as the instance member. In the following example the "B.a = "hiding b.a" replaces the reference to the member descriptor for instance member 'a' in the type dict. This eliminates access to all instance members of that name throughout the application. >>> class B(object): ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 >>> B.a = "hiding b.a" >>> b.a 'hiding b.a' >>> b.a = 5 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-06 09:43 Message: Logged In: YES user_id=31435 Changed category, and assigned to me in Guido's absence. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 From noreply@sourceforge.net Tue Nov 6 18:47:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 10:47:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Open Resolution: Works For Me Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 10:47 Message: Logged In: YES user_id=356603 To answer your question, I am using 2.2b1, from the tarball not from CVS. In the second example, I get SystemError from the first call to add actually. I thought it was from the second one, but since I changed from my own exception class to Exception it became hidden. When I remove the uninteded self parameter of add, I get SystemError in the second call, as I thought I was getting. But you get TypeError.. hmmm.. I suppose I should download the CVS then?.. Or maybe wait for 2.2b2.. Sverker ps. Here's my updated second example, anyway.. import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(self, x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:33 Message: Logged In: YES user_id=3066 I cannot reproduce this using either code snippet. Note that the second example should (and does for me) raise TypeError when calling add() -- you left in a "self" that appears to be unintended. Are you using 2.2b1 or CVS python? ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Tue Nov 6 19:07:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 11:07:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: Works For Me Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 11:07 Message: Logged In: YES user_id=356603 Ooops, in the example I sent you I hadnt removed the self parameter after all. I mixed up something. But when I did remove the self parameter it went as I said. Actually, I get a segmentation fault running it directly from the shell. (When running it from Emacs mode or importing it I get SystemError.) Sverker Shell transcript: nicosys [228] python defenvbug3.py first add ok Segmentation fault nicosys [229] cat defenvbug3.py import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. nicosys [230] python Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import defenvbug3.py first add ok Traceback (most recent call last): File "", line 1, in ? SystemError: NULL result without error in PyObject_Call >>> nicosys [231] whence python /usr/local/bin/python is actually /hda7/local/bin/python /usr/local/bin/python: ELF 32-bit LSB executable i386 (386 and up) Version 1 nicosys [232] ldd /hda7/local/bin/python libdl.so.2 => /lib/libdl.so.2 (0x40018000) libpthread.so.0 => /lib/libpthread.so.0 (0x4001d000) libutil.so.1 => /lib/libutil.so.1 (0x40030000) libm.so.6 => /lib/libm.so.6 (0x40033000) libc.so.6 => /lib/libc.so.6 (0x40052000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) nicosys [233] ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 10:47 Message: Logged In: YES user_id=356603 To answer your question, I am using 2.2b1, from the tarball not from CVS. In the second example, I get SystemError from the first call to add actually. I thought it was from the second one, but since I changed from my own exception class to Exception it became hidden. When I remove the uninteded self parameter of add, I get SystemError in the second call, as I thought I was getting. But you get TypeError.. hmmm.. I suppose I should download the CVS then?.. Or maybe wait for 2.2b2.. Sverker ps. Here's my updated second example, anyway.. import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(self, x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:33 Message: Logged In: YES user_id=3066 I cannot reproduce this using either code snippet. Note that the second example should (and does for me) raise TypeError when calling add() -- you left in a "self" that appears to be unintended. Are you using 2.2b1 or CVS python? ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Tue Nov 6 19:13:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 11:13:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific >Status: Open Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Martin v. Löwis (loewis) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-06 11:12 Message: Logged In: YES user_id=21627 The likely cause is that sbsize_t is not defined. Please try to find out what header file of HP-UX defines sbsize_t, and under what conditions? Outright including that header file probably is the wrong solution, so please also find out where it is included from, tracking it eventually back to socket.h. It appears that a certain #define is missing to cause a proper definition of sbsize_t. We must find out what that define is, and why it is not set when compiling with gcc. Also, could you please identify yourself? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-06 04:42 Message: Logged In: NO gcc --version is now 3.0.1 The problem however is still there: building '_socket' extension gcc -g -O2 -Wall -Wstrict-prototypes -fpic -I. -I/utmnt/ut/si/dv0216/Python-2.1.1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.1/socketmodule.o In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:145: /usr/include/sys/socket.h:439: parse error before "sendfile" /usr/include/sys/socket.h:439: parse error before "bsize_t" /usr/include/sys/socket.h:441: parse error before "sendpath" /usr/include/sys/socket.h:441: parse error before "bsize_t" /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_accept': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:795: warning: passing arg 3 of `accept' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getsockopt': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:978: warning: passing arg 5 of `getsockopt' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:992: warning: passing arg 5 of `getsockopt' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getsockname': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1188: warning: passing arg 3 of `getsockname' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getpeername': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1218: warning: passing arg 3 of `getpeername' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_recvfrom': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1376: warning: passing arg 6 of `recvfrom' from incompatible pointer type WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 Here are the offending lines from socket.h extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); However, my medium c skills tell me that the lines are ok. I cannot offer an explanation why gcc fails here. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 08:57 Message: Logged In: YES user_id=21627 gcc 2.95 does not support HP-UX 11; this appears to be an instance of that problem. The __va__list problem is not at all Python-specific; it occurs in many packages (just search Google for "HP-UX __va__list"). With such problems, there is no point into looking further into the problem; just drop the compiler. If you want to port Python to HP-UX and gcc 2.95, you are on your own. Notice that gcc 3.0.1 *does* support HP-UX 11; I recommend to upgrade. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 07:41 Message: Logged In: NO gcc version is 2.95.3 HP-UX hp18 B.11.11 U 9000/785 2011589119 both Python 2.1.1 and 2.2b1 do not compile ootb here. Error messages during compile are building '_socket' extension gcc -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/utmnt/ut/si/dv0216/Python-2.2b1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.2/socketmodule.o In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:42, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: `__va__list' redefined /usr/include/sys/stdsyms.h:422: warning: this is the location of the previous definition In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:47, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/string.h:26: warning: `__va__list' redefined /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: this is the location of the previous definition In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:152: /usr/include/sys/socket.h:439: parse error before `sendfile' /usr/include/sys/socket.h:439: parse error before `bsize_t' /usr/include/sys/socket.h:440: warning: data definition has no type or storage class /usr/include/sys/socket.h:441: parse error before `sendpath' /usr/include/sys/socket.h:441: parse error before `bsize_t' /usr/include/sys/socket.h:442: warning: data definition has no type or storage class /usr/include/sys/socket.h:456: parse error before `__sendfile64' /usr/include/sys/socket.h:456: parse error before `bsize_t' /usr/include/sys/socket.h:456: warning: data definition has no type or storage class /usr/include/sys/socket.h:457: parse error before `__sendpath64' /usr/include/sys/socket.h:457: parse error before `bsize_t' /usr/include/sys/socket.h:457: warning: data definition has no type or storage class /usr/include/sys/socket.h:459: parse error before `sendfile' /usr/include/sys/socket.h:459: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendfile': /usr/include/sys/socket.h:459: parse error before `bsize_t' /usr/include/sys/socket.h: At top level: /usr/include/sys/socket.h:460: parse error before `sendpath' /usr/include/sys/socket.h:460: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendpath': /usr/include/sys/socket.h:460: parse error before `bsize_t' In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:238: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: At top level: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:203: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:212: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `freeaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:219: warning: implicit declaration of function `free' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `str_isnumber': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:231: warning: subscript has type `char' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `getaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:345: warning: implicit declaration of function `atoi' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:396: warning: implicit declaration of function `malloc' WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 I now thought that since the errors happen in a system header that compiles with cc and not gcc, why not copy to "socket.h" and delete the two offending lines. However, as you see, is still included (why ?). Another option is using cc just for this library, but I'm not sure, that the object files are compatible. Using cc alone creates a lot of non-working modules, although it is in (extended) ANSI mode. I also tried the HP Porting centre (http://hpux.asknet.de) but the Python 2.1 package there -just like mine- does crash on importing socket (too with 2.2b1) hp18: Python-2.2b1 % ./python Python 2.2b1 (#4, Nov 1 2001, 16:36:54) [C] on hp-uxB Type "help", "copyright", "credits" or "license" for more information. >>> import socket Traceback (most recent call last): File "", line 1, in ? File "/utmnt/ut/si/dv0216/Python-2.2b1/Lib/socket.py", line 41, in ? from _socket import * ImportError: No module named _socket >>> ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-30 00:56 Message: Logged In: YES user_id=21627 It would be helpful if you would provide all details in a report also. When you say "it crashes", did you really mean "it crashes" (i.e. with a core dump, machine reboot, or the like)? If the compiler merely rejected the code, it would be good to see what the error message was. Please also report version numbers: HP-UX version, gcc version, etc. Are you sure you are using a gcc built for your OS version? If the gcc doesn't get prototypes right, this often comes from having fixincluded header files of a different OS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 04:59 Message: Logged In: YES user_id=6380 Please try again with Python 2.2b1 (http://python.org/2.2/). We've done a lot of work and we've got at least one report that it now works out of the box on HP-UX 11. I'd like to hear if that solves your problems (and if 2.2b1 builds with either compiler). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Tue Nov 6 20:39:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 12:39:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-478851 ] Generates KeyboardInterrupt in bg Message-ID: Bugs item #478851, was opened at 2001-11-06 12:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478851&group_id=5470 Category: Macintosh Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jack Jansen (jackjansen) Assigned to: Jack Jansen (jackjansen) Summary: Generates KeyboardInterrupt in bg Initial Comment: The code in MacPython to check for command-period looks in the low-level event queue, but ignores whether MacPython is in the foreground or not. Therefore interrupting any program will potentially also interrupt a MacPython running in the background. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478851&group_id=5470 From noreply@sourceforge.net Tue Nov 6 20:45:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 12:45:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-478428 ] dbm build fails Message-ID: Bugs item #478428, was opened at 2001-11-05 12:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478428&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: John Buell (dadaist) Assigned to: Jack Jansen (jackjansen) Summary: dbm build fails Initial Comment: Trying a build with cvs source downloaded this morning. I've been sure to do the configure --with-suffix=_ on case-insensitive Mac OS X (10.1), but I'm having some problem with the 'dbm' module. Here's a copy of the messages: building 'dbm' extension /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c: In function `dbm_subscript': /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c:104: warning: implicit declaration of function `dbm_error' /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c:105: warning: implicit declaration of function `dbm_clearerr' cc -g -O3 -Wall -Wstrict-prototypes -no-cpp-precomp -I. -I/Users/jbuell/Documents/Python/python/dist/src/./Include -I/Users/jbuell/Documents/Python/python/dist/src/./Mac/Include -I/usr/local/include -IInclude/ -c /Users/jbuell/Documents/Python/python/dist/src/Modules/dbmmodule.c -o build/temp.darwin-1.4-Power Macintosh-2.2/dbmmodule.o cc -bundle -flat_namespace -undefined suppress build/temp.darwin-1.4-Power Macintosh-2.2/dbmmodule.o -L/usr/local/lib -o build/lib.darwin-1.4-Power Macintosh-2.2/dbm.so building 'gdbm' extension dyld: ./python_ Undefined symbols: __ashldi3 make: *** [sharedmods] Error 67 Any suggestions? Anything I can change in my settings to get past this? -John ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-11-06 12:45 Message: Logged In: YES user_id=45365 Note that the problem is with gdbm, not with dbm. Dbm seems to build fine (with a warning). I've heard a report of this before. Needless to say, for me there is no problem whatsoever:-) I think that the problem is that setup.py detects files that indicate to it that dbm is available while in fact it is not. Could you check whether you have a libgdbm.a and/or a gdbm.h on your system somewhere? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478428&group_id=5470 From noreply@sourceforge.net Tue Nov 6 22:14:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 14:14:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-478887 ] imaplib patch Message-ID: Bugs item #478887, was opened at 2001-11-06 14:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478887&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: imaplib patch Initial Comment: The comment in the patch says it all. I am using Linux (various distros/versions) with Jython 2.1a3. *** /usr/lib/python2.1/imaplib.py Sat Oct 20 10:48:35 2001 --- lib/modules/imaplib.py Tue Nov 6 16:59:15 2001 *************** *** 198,204 **** """Setup 'self.sock' and 'self.file'.""" self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect((self.host, self.port)) ! self.file = self.sock.makefile('r') def recent(self): --- 198,206 ---- """Setup 'self.sock' and 'self.file'.""" self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect((self.host, self.port)) ! # use binary mode -- with jython (java) not specifying binary mode ! # causes a CRLF->LF translation and screws up the data later on ! self.file = self.sock.makefile('rb') def recent(self): ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478887&group_id=5470 From noreply@sourceforge.net Tue Nov 6 23:50:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 15:50:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-477999 ] win32ui.FullPath() fails in PythonWin Message-ID: Bugs item #477999, was opened at 2001-11-04 06:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 Category: Windows Group: Python 2.1.1 >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: win32ui.FullPath() fails in PythonWin Initial Comment: A line in a method of a Class, like the following: self.name = win32ui.FullPath(self.name) works fine when a script is run from the command line. However, it fails when run in PythonWin. From the command line, the full path is set for a filename. In PythonWin, a filename is always shown/returned as being in the root directory of a drive. Using: ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 and the PythonWin that came with this release. ---------------------------------------------------------------------- >Comment By: Mark Hammond (mhammond) Date: 2001-11-06 15:50 Message: Logged In: YES user_id=14198 win32ui.FullPath uses the cwd to determine the full path - this is by design. I agree Pythonwin should be smarter about the cwd for the script, and indeed the next version will use the scripts directory as the cwd. Later versions may also allow this to be specified in the "Run" dialog. Resolving as not a bug. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 10:48 Message: Logged In: YES user_id=31435 Reassigned to MarkH. Anonymous, you should really report this to ActiveState. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-04 07:16 Message: Logged In: NO This might not be a bug. Apparently PythonWin does not default to the directory of the script that is being run. When I close PythonWin and restart it in the directory of the script that is being run, win32ui.FullPath() works. I don't know if this is intentional behavior or not. It it is not, I suggest that the directory of the script in the "Run Script" dialog of PythonWin be the current/default directory. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477999&group_id=5470 From noreply@sourceforge.net Wed Nov 7 02:06:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 18:06:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-478949 ] Windows installer start menu registry Message-ID: Bugs item #478949, was opened at 2001-11-06 18:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bruce Sherwood (bsherwood) Assigned to: Nobody/Anonymous (nobody) Summary: Windows installer start menu registry Initial Comment: The new installer for Windows Python 2.2b1 enters a long string into the InstallGroup portion of the registry: "C:\WINDOWS\StartMenu\Programs\Python 2.2" Previous Windows installers entered a short string: "Python 2.1" I believe the long string is incorrect and the short string correct. For example, when I try to use the freeware installer InnoSetup (plus extensions) to create an installer for VPython, an incorrect start menu address is generated when I attempt to install a on the start menu a pointer to the idlefork version of IDLE. This worked fine in the past, with the short form registry item. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 From noreply@sourceforge.net Wed Nov 7 04:34:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 20:34:38 -0800 Subject: [Python-bugs-list] [ python-Bugs-478949 ] Windows installer start menu registry Message-ID: Bugs item #478949, was opened at 2001-11-06 18:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bruce Sherwood (bsherwood) >Assigned to: Tim Peters (tim_one) Summary: Windows installer start menu registry Initial Comment: The new installer for Windows Python 2.2b1 enters a long string into the InstallGroup portion of the registry: "C:\WINDOWS\StartMenu\Programs\Python 2.2" Previous Windows installers entered a short string: "Python 2.1" I believe the long string is incorrect and the short string correct. For example, when I try to use the freeware installer InnoSetup (plus extensions) to create an installer for VPython, an incorrect start menu address is generated when I attempt to install a on the start menu a pointer to the idlefork version of IDLE. This worked fine in the past, with the short form registry item. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-06 20:34 Message: Logged In: YES user_id=31435 Exactly which key are you talking about? Please give a full path, starting with the hive (HKLM or HKCU or HKCR?). "The InstallGroup portion of the registry" doesn't mean much to me, so unless you spell it out I can only guess. Wise did change how they define their internal %GROUP% vrbl between versions 5.0a (used for pre-2.2 installers) and 8.14 (used for 2.2), and in the way you describe. This is used for several purposes, though, and some clearly need the full path. As to correct vs incorrect, beats me -- Python sets some registry entries common to all Windows programs, and "correct" for them means they work the way MS says they should; I don't know of any we get wrong in that sense. Other registry entries are unique to Python, and since neither Windows nor Python ever looks at them, I don't know why we even set them up (beyond that, AFAIK, we always have). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 From noreply@sourceforge.net Wed Nov 7 07:00:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 06 Nov 2001 23:00:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-478764 ] does not install on Windows XP Message-ID: Bugs item #478764, was opened at 2001-11-06 08:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 Category: Installation Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: does not install on Windows XP Initial Comment: When executing the BeOpen-Python-2.0.exe installation file, I get the following error from the installer: Error: Could not write file, your disk may be full. The drive I executed the file on has 143GB free. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-06 23:00 Message: Logged In: NO Oops. Go ahead and toast this bug -- turns out my TEMP environment variable was set to a drive that was full. I thought I had it set to a drive that had 146GB free. Sorry for the mix-up. -matt ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-06 09:37 Message: Logged In: YES user_id=31435 Were you logged in to an Administrator account? If not, try that. If so, this may be hopeless: the 2.0 installer was built with a very old version of Wise, written before even Win98 existed, and is a 16-bit program that started having problems on Win2000 already. Ditto the 2.1 installer. The 2.2 installer is built with a recent version of Wise, and is a 32-bit program. We don't have XP, though, so can't say for sure. You may also have better luck using ActiveState's distribution (which uses different installation technology). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 From noreply@sourceforge.net Wed Nov 7 08:32:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 00:32:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-478003 ] possible memory leaks in err handling Message-ID: Bugs item #478003, was opened at 2001-11-04 07:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478003&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Martin v. Löwis (loewis) Summary: possible memory leaks in err handling Initial Comment: This patch fixes problems caught during code inspection they seem correct, but I don't think the regression tests hit these paths. I'm not sure how to test them. Files in patch: Modules/getaddrinfo.c Modules/socketmodule.c Python/compile.c ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 00:32 Message: Logged In: YES user_id=21627 I just applied the missing parts in a slightly massaged form, as getaddrinfo.c 1.9 and socketmodule.c 1.194. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 11:27 Message: Logged In: YES user_id=31435 I applied (just) the compile.c part of the patch (thanks!), in Python/compile.c; new revision: 2.228 Reassigned to MvL for the getaddrinfo and socketmodule parts. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478003&group_id=5470 From noreply@sourceforge.net Wed Nov 7 08:35:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 00:35:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-478764 ] does not install on Windows XP Message-ID: Bugs item #478764, was opened at 2001-11-06 08:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 Category: Installation Group: Platform-specific >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: does not install on Windows XP Initial Comment: When executing the BeOpen-Python-2.0.exe installation file, I get the following error from the installer: Error: Could not write file, your disk may be full. The drive I executed the file on has 143GB free. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 00:35 Message: Logged In: YES user_id=21627 Report withdrawn by submitter. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-06 23:00 Message: Logged In: NO Oops. Go ahead and toast this bug -- turns out my TEMP environment variable was set to a drive that was full. I thought I had it set to a drive that had 146GB free. Sorry for the mix-up. -matt ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-06 09:37 Message: Logged In: YES user_id=31435 Were you logged in to an Administrator account? If not, try that. If so, this may be hopeless: the 2.0 installer was built with a very old version of Wise, written before even Win98 existed, and is a 16-bit program that started having problems on Win2000 already. Ditto the 2.1 installer. The 2.2 installer is built with a recent version of Wise, and is a 32-bit program. We don't have XP, though, so can't say for sure. You may also have better luck using ActiveState's distribution (which uses different installation technology). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478764&group_id=5470 From noreply@sourceforge.net Wed Nov 7 14:44:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 06:44:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-478949 ] Windows installer start menu registry Message-ID: Bugs item #478949, was opened at 2001-11-06 18:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bruce Sherwood (bsherwood) Assigned to: Tim Peters (tim_one) Summary: Windows installer start menu registry Initial Comment: The new installer for Windows Python 2.2b1 enters a long string into the InstallGroup portion of the registry: "C:\WINDOWS\StartMenu\Programs\Python 2.2" Previous Windows installers entered a short string: "Python 2.1" I believe the long string is incorrect and the short string correct. For example, when I try to use the freeware installer InnoSetup (plus extensions) to create an installer for VPython, an incorrect start menu address is generated when I attempt to install a on the start menu a pointer to the idlefork version of IDLE. This worked fine in the past, with the short form registry item. ---------------------------------------------------------------------- >Comment By: Bruce Sherwood (bsherwood) Date: 2001-11-07 06:44 Message: Logged In: YES user_id=34881 Sorry; I thought "InstallGroup" was adequate ID. It's in HKEY_LOCAL_MACHINE, Python, PythonCore, 2.2, InstallPath, InstallGroup. >From your explanation, I see that this is a feature, not a bug. The rule then is that starting with Python 2.2, the InstallGroup entry will be the full path name to the location of the start menu entry rather than simply "Python N.N" as it was in the past. I just need to change my own installer to expect the full path name in the future. I made the wrong guess in calling it "incorrect" -- it's just different, not incorrect. Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-06 20:34 Message: Logged In: YES user_id=31435 Exactly which key are you talking about? Please give a full path, starting with the hive (HKLM or HKCU or HKCR?). "The InstallGroup portion of the registry" doesn't mean much to me, so unless you spell it out I can only guess. Wise did change how they define their internal %GROUP% vrbl between versions 5.0a (used for pre-2.2 installers) and 8.14 (used for 2.2), and in the way you describe. This is used for several purposes, though, and some clearly need the full path. As to correct vs incorrect, beats me -- Python sets some registry entries common to all Windows programs, and "correct" for them means they work the way MS says they should; I don't know of any we get wrong in that sense. Other registry entries are unique to Python, and since neither Windows nor Python ever looks at them, I don't know why we even set them up (beyond that, AFAIK, we always have). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 From noreply@sourceforge.net Wed Nov 7 15:16:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 07:16:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-478949 ] Windows installer start menu registry Message-ID: Bugs item #478949, was opened at 2001-11-06 18:05 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 Category: Installation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Bruce Sherwood (bsherwood) Assigned to: Tim Peters (tim_one) Summary: Windows installer start menu registry Initial Comment: The new installer for Windows Python 2.2b1 enters a long string into the InstallGroup portion of the registry: "C:\WINDOWS\StartMenu\Programs\Python 2.2" Previous Windows installers entered a short string: "Python 2.1" I believe the long string is incorrect and the short string correct. For example, when I try to use the freeware installer InnoSetup (plus extensions) to create an installer for VPython, an incorrect start menu address is generated when I attempt to install a on the start menu a pointer to the idlefork version of IDLE. This worked fine in the past, with the short form registry item. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-07 07:16 Message: Logged In: YES user_id=31435 Thanks for the followup, Bruce! This is one of the registry entries Windows never looks at, and core Python doesn't either, so the only definition of "correct" is "looks like whatever the heck we used to do". Don't change your installer! I've changed ours instead, to use a different variable here -- it will be "Python 2.2" in 2.2b2. One caution: starting with Win2000, an ordinary user doesn't have enough privilege to write into the HKEY_LOCAL_MACHINE hive, including our installer run by an ordinary user. If our installer detects that, *or* if the user selects the new "Non-Admin install" option, all these keys get written under HKEY_CURRENT_USER\etc instead. ---------------------------------------------------------------------- Comment By: Bruce Sherwood (bsherwood) Date: 2001-11-07 06:44 Message: Logged In: YES user_id=34881 Sorry; I thought "InstallGroup" was adequate ID. It's in HKEY_LOCAL_MACHINE, Python, PythonCore, 2.2, InstallPath, InstallGroup. >From your explanation, I see that this is a feature, not a bug. The rule then is that starting with Python 2.2, the InstallGroup entry will be the full path name to the location of the start menu entry rather than simply "Python N.N" as it was in the past. I just need to change my own installer to expect the full path name in the future. I made the wrong guess in calling it "incorrect" -- it's just different, not incorrect. Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-06 20:34 Message: Logged In: YES user_id=31435 Exactly which key are you talking about? Please give a full path, starting with the hive (HKLM or HKCU or HKCR?). "The InstallGroup portion of the registry" doesn't mean much to me, so unless you spell it out I can only guess. Wise did change how they define their internal %GROUP% vrbl between versions 5.0a (used for pre-2.2 installers) and 8.14 (used for 2.2), and in the way you describe. This is used for several purposes, though, and some clearly need the full path. As to correct vs incorrect, beats me -- Python sets some registry entries common to all Windows programs, and "correct" for them means they work the way MS says they should; I don't know of any we get wrong in that sense. Other registry entries are unique to Python, and since neither Windows nor Python ever looks at them, I don't know why we even set them up (beyond that, AFAIK, we always have). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478949&group_id=5470 From noreply@sourceforge.net Wed Nov 7 15:17:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 07:17:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-479137 ] unset PYTHON* shell vars Message-ID: Bugs item #479137, was opened at 2001-11-07 07:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479137&group_id=5470 Category: Build Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: unset PYTHON* shell vars Initial Comment: When I install on a new machine, MacOS X iBook using bash, with PYTHONHOME, PYTHONPATH, and PYTHONSTARTUP set the make, test, and install will fail. Usually this is not a problem since I have previous versions of Python and I have never seen this. I would suggest that in make these variables are unset before continuing. It took me a while to figure out why various standard packages (os, regex, etc) were not being found. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479137&group_id=5470 From noreply@sourceforge.net Wed Nov 7 17:14:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 09:14:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-479186 ] compiler generates bad code for "del" Message-ID: Bugs item #479186, was opened at 2001-11-07 09:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479186&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Neil Schemenauer (nascheme) Assigned to: Jeremy Hylton (jhylton) Summary: compiler generates bad code for "del" Initial Comment: The compiler generates bad code from the following statement: del a[0], a[1] Here is the AST: Stmt([AssTuple([Subscript(Name('a'), 'OP_DELETE', [Const(0)]), Subscript(Name('a'), 'OP_DELETE', [Const(1)])])]) The bytecode generated is: 0 SET_LINENO 0 3 UNPACK_SEQUENCE 2 BAD ------^^^^^^^^^^^^^^^^^^^^^^^^^^ 6 SET_LINENO 2 9 LOAD_NAME 0 (a) 12 LOAD_CONST 1 (0) 15 DELETE_SUBSCR 16 LOAD_NAME 0 (a) 19 LOAD_CONST 2 (1) 22 DELETE_SUBSCR The problem, AFAICT, is that findOp does not find OP_DELETE. Adding a visitSubscript method to OpFinder seems to be the fix. A small patch is attached. This is a 2.1.2 bugfix candidate, IMO. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479186&group_id=5470 From noreply@sourceforge.net Wed Nov 7 17:18:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 09:18:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-479137 ] unset PYTHON* shell vars Message-ID: Bugs item #479137, was opened at 2001-11-07 07:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479137&group_id=5470 Category: Build Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: unset PYTHON* shell vars Initial Comment: When I install on a new machine, MacOS X iBook using bash, with PYTHONHOME, PYTHONPATH, and PYTHONSTARTUP set the make, test, and install will fail. Usually this is not a problem since I have previous versions of Python and I have never seen this. I would suggest that in make these variables are unset before continuing. It took me a while to figure out why various standard packages (os, regex, etc) were not being found. ---------------------------------------------------------------------- >Comment By: Neil Schemenauer (nascheme) Date: 2001-11-07 09:18 Message: Logged In: YES user_id=35752 This is fixed in 2.2 with the addition of the -E command line option (ignore environment variables like PYTHONPATH and PYTHONHOME that modify the behavior of the interpreter). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479137&group_id=5470 From noreply@sourceforge.net Wed Nov 7 17:42:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 09:42:28 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: Works For Me Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:42 Message: Logged In: YES user_id=356603 I have got the cvs version now. It also gives the error or segmentation fault. Either compiled with Py_Debug enabled in configuration, or with another version of my test program. (The Py_Debug definition changes what Python does in eval_frame so the error is revealed in the original test. ) See attached files: wrbug4.py, new test program. wrbug4.mail, some info. wrbug4.gdb-transcript, a raw gdb transcript... /Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 11:07 Message: Logged In: YES user_id=356603 Ooops, in the example I sent you I hadnt removed the self parameter after all. I mixed up something. But when I did remove the self parameter it went as I said. Actually, I get a segmentation fault running it directly from the shell. (When running it from Emacs mode or importing it I get SystemError.) Sverker Shell transcript: nicosys [228] python defenvbug3.py first add ok Segmentation fault nicosys [229] cat defenvbug3.py import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. nicosys [230] python Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import defenvbug3.py first add ok Traceback (most recent call last): File "", line 1, in ? SystemError: NULL result without error in PyObject_Call >>> nicosys [231] whence python /usr/local/bin/python is actually /hda7/local/bin/python /usr/local/bin/python: ELF 32-bit LSB executable i386 (386 and up) Version 1 nicosys [232] ldd /hda7/local/bin/python libdl.so.2 => /lib/libdl.so.2 (0x40018000) libpthread.so.0 => /lib/libpthread.so.0 (0x4001d000) libutil.so.1 => /lib/libutil.so.1 (0x40030000) libm.so.6 => /lib/libm.so.6 (0x40033000) libc.so.6 => /lib/libc.so.6 (0x40052000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) nicosys [233] ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 10:47 Message: Logged In: YES user_id=356603 To answer your question, I am using 2.2b1, from the tarball not from CVS. In the second example, I get SystemError from the first call to add actually. I thought it was from the second one, but since I changed from my own exception class to Exception it became hidden. When I remove the uninteded self parameter of add, I get SystemError in the second call, as I thought I was getting. But you get TypeError.. hmmm.. I suppose I should download the CVS then?.. Or maybe wait for 2.2b2.. Sverker ps. Here's my updated second example, anyway.. import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(self, x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:33 Message: Logged In: YES user_id=3066 I cannot reproduce this using either code snippet. Note that the second example should (and does for me) raise TypeError when calling add() -- you left in a "self" that appears to be unintended. Are you using 2.2b1 or CVS python? ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Wed Nov 7 17:50:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 09:50:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: Works For Me Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:50 Message: Logged In: YES user_id=356603 wrbug4.gdb-transcript contains a raw gdb transcript, in case it can be of some use / Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:47 Message: Logged In: YES user_id=356603 I couldnt figure out how to upload several files with the same comment. Well, this is to tell you that wrbug4.mail contains some more info. /Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:42 Message: Logged In: YES user_id=356603 I have got the cvs version now. It also gives the error or segmentation fault. Either compiled with Py_Debug enabled in configuration, or with another version of my test program. (The Py_Debug definition changes what Python does in eval_frame so the error is revealed in the original test. ) See attached files: wrbug4.py, new test program. wrbug4.mail, some info. wrbug4.gdb-transcript, a raw gdb transcript... /Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 11:07 Message: Logged In: YES user_id=356603 Ooops, in the example I sent you I hadnt removed the self parameter after all. I mixed up something. But when I did remove the self parameter it went as I said. Actually, I get a segmentation fault running it directly from the shell. (When running it from Emacs mode or importing it I get SystemError.) Sverker Shell transcript: nicosys [228] python defenvbug3.py first add ok Segmentation fault nicosys [229] cat defenvbug3.py import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. nicosys [230] python Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import defenvbug3.py first add ok Traceback (most recent call last): File "", line 1, in ? SystemError: NULL result without error in PyObject_Call >>> nicosys [231] whence python /usr/local/bin/python is actually /hda7/local/bin/python /usr/local/bin/python: ELF 32-bit LSB executable i386 (386 and up) Version 1 nicosys [232] ldd /hda7/local/bin/python libdl.so.2 => /lib/libdl.so.2 (0x40018000) libpthread.so.0 => /lib/libpthread.so.0 (0x4001d000) libutil.so.1 => /lib/libutil.so.1 (0x40030000) libm.so.6 => /lib/libm.so.6 (0x40033000) libc.so.6 => /lib/libc.so.6 (0x40052000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) nicosys [233] ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 10:47 Message: Logged In: YES user_id=356603 To answer your question, I am using 2.2b1, from the tarball not from CVS. In the second example, I get SystemError from the first call to add actually. I thought it was from the second one, but since I changed from my own exception class to Exception it became hidden. When I remove the uninteded self parameter of add, I get SystemError in the second call, as I thought I was getting. But you get TypeError.. hmmm.. I suppose I should download the CVS then?.. Or maybe wait for 2.2b2.. Sverker ps. Here's my updated second example, anyway.. import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(self, x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:33 Message: Logged In: YES user_id=3066 I cannot reproduce this using either code snippet. Note that the second example should (and does for me) raise TypeError when calling add() -- you left in a "self" that appears to be unintended. Are you using 2.2b1 or CVS python? ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Wed Nov 7 17:47:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 09:47:21 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: Works For Me Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:47 Message: Logged In: YES user_id=356603 I couldnt figure out how to upload several files with the same comment. Well, this is to tell you that wrbug4.mail contains some more info. /Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:42 Message: Logged In: YES user_id=356603 I have got the cvs version now. It also gives the error or segmentation fault. Either compiled with Py_Debug enabled in configuration, or with another version of my test program. (The Py_Debug definition changes what Python does in eval_frame so the error is revealed in the original test. ) See attached files: wrbug4.py, new test program. wrbug4.mail, some info. wrbug4.gdb-transcript, a raw gdb transcript... /Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 11:07 Message: Logged In: YES user_id=356603 Ooops, in the example I sent you I hadnt removed the self parameter after all. I mixed up something. But when I did remove the self parameter it went as I said. Actually, I get a segmentation fault running it directly from the shell. (When running it from Emacs mode or importing it I get SystemError.) Sverker Shell transcript: nicosys [228] python defenvbug3.py first add ok Segmentation fault nicosys [229] cat defenvbug3.py import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. nicosys [230] python Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import defenvbug3.py first add ok Traceback (most recent call last): File "", line 1, in ? SystemError: NULL result without error in PyObject_Call >>> nicosys [231] whence python /usr/local/bin/python is actually /hda7/local/bin/python /usr/local/bin/python: ELF 32-bit LSB executable i386 (386 and up) Version 1 nicosys [232] ldd /hda7/local/bin/python libdl.so.2 => /lib/libdl.so.2 (0x40018000) libpthread.so.0 => /lib/libpthread.so.0 (0x4001d000) libutil.so.1 => /lib/libutil.so.1 (0x40030000) libm.so.6 => /lib/libm.so.6 (0x40033000) libc.so.6 => /lib/libc.so.6 (0x40052000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) nicosys [233] ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 10:47 Message: Logged In: YES user_id=356603 To answer your question, I am using 2.2b1, from the tarball not from CVS. In the second example, I get SystemError from the first call to add actually. I thought it was from the second one, but since I changed from my own exception class to Exception it became hidden. When I remove the uninteded self parameter of add, I get SystemError in the second call, as I thought I was getting. But you get TypeError.. hmmm.. I suppose I should download the CVS then?.. Or maybe wait for 2.2b2.. Sverker ps. Here's my updated second example, anyway.. import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(self, x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:33 Message: Logged In: YES user_id=3066 I cannot reproduce this using either code snippet. Note that the second example should (and does for me) raise TypeError when calling add() -- you left in a "self" that appears to be unintended. Are you using 2.2b1 or CVS python? ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Wed Nov 7 17:55:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 09:55:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-478534 ] SystemError with WeakKeyDictionary Message-ID: Bugs item #478534, was opened at 2001-11-05 19:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: Works For Me Priority: 5 Submitted By: Sverker Nilsson (svenil) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SystemError with WeakKeyDictionary Initial Comment: SystemError with WeakKeyDictionary A SystemError is generated when trying to iterate over a function returned from a function that stored it in a WeakKeyDictionary. The expected error was TypeError. Sverker Nilsson Examples: This program gives a SystemError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f for x in encapsulate(): print x Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. ... Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 9, in ? for x in encapsulate(): SystemError: error return without exception set >>> A variation of it, with a temporary variable, gives the expected TypeError: import weakref ref = weakref.WeakKeyDictionary() def encapsulate(): f = lambda : () ref[f] = None return f g = encapsulate() for x in g: print x Traceback (most recent call last): File "", line 1, in ? File "/tmp/pythona04442", line 10, in ? for x in g: TypeError: iteration over non-sequence >>> ---------------------------------------------------------------------- >Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:55 Message: Logged In: YES user_id=356603 Seems the files didnt make it, sorry for the fuzz.. trying again. S. ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:50 Message: Logged In: YES user_id=356603 wrbug4.gdb-transcript contains a raw gdb transcript, in case it can be of some use / Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:47 Message: Logged In: YES user_id=356603 I couldnt figure out how to upload several files with the same comment. Well, this is to tell you that wrbug4.mail contains some more info. /Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-07 09:42 Message: Logged In: YES user_id=356603 I have got the cvs version now. It also gives the error or segmentation fault. Either compiled with Py_Debug enabled in configuration, or with another version of my test program. (The Py_Debug definition changes what Python does in eval_frame so the error is revealed in the original test. ) See attached files: wrbug4.py, new test program. wrbug4.mail, some info. wrbug4.gdb-transcript, a raw gdb transcript... /Sverker ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 11:07 Message: Logged In: YES user_id=356603 Ooops, in the example I sent you I hadnt removed the self parameter after all. I mixed up something. But when I did remove the self parameter it went as I said. Actually, I get a segmentation fault running it directly from the shell. (When running it from Emacs mode or importing it I get SystemError.) Sverker Shell transcript: nicosys [228] python defenvbug3.py first add ok Segmentation fault nicosys [229] cat defenvbug3.py import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. nicosys [230] python Python 2.2b1 (#5, Oct 20 2001, 03:03:53) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import defenvbug3.py first add ok Traceback (most recent call last): File "", line 1, in ? SystemError: NULL result without error in PyObject_Call >>> nicosys [231] whence python /usr/local/bin/python is actually /hda7/local/bin/python /usr/local/bin/python: ELF 32-bit LSB executable i386 (386 and up) Version 1 nicosys [232] ldd /hda7/local/bin/python libdl.so.2 => /lib/libdl.so.2 (0x40018000) libpthread.so.0 => /lib/libpthread.so.0 (0x4001d000) libutil.so.1 => /lib/libutil.so.1 (0x40030000) libm.so.6 => /lib/libm.so.6 (0x40033000) libc.so.6 => /lib/libc.so.6 (0x40052000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) nicosys [233] ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-06 10:47 Message: Logged In: YES user_id=356603 To answer your question, I am using 2.2b1, from the tarball not from CVS. In the second example, I get SystemError from the first call to add actually. I thought it was from the second one, but since I changed from my own exception class to Exception it became hidden. When I remove the uninteded self parameter of add, I get SystemError in the second call, as I thought I was getting. But you get TypeError.. hmmm.. I suppose I should download the CVS then?.. Or maybe wait for 2.2b2.. Sverker ps. Here's my updated second example, anyway.. import weakref ref = weakref.WeakKeyDictionary() class MyError(Exception): pass def add(self, x): raise MyError def encapsulate(): f = lambda : () ref[f] = None return f try: add(encapsulate()) except MyError: pass print 'first add ok' add(encapsulate()) # Here we would like to get Exception but gets SystemError instead. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-06 08:33 Message: Logged In: YES user_id=3066 I cannot reproduce this using either code snippet. Note that the second example should (and does for me) raise TypeError when calling add() -- you left in a "self" that appears to be unintended. Are you using 2.2b1 or CVS python? ---------------------------------------------------------------------- Comment By: Sverker Nilsson (svenil) Date: 2001-11-05 23:17 Message: Logged In: YES user_id=356603 The same problem now occured in another situation. I am enclosing the condensed program. /Sverker ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478534&group_id=5470 From noreply@sourceforge.net Wed Nov 7 20:08:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 12:08:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Wed Nov 7 22:01:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 14:01:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-478887 ] imaplib patch Message-ID: Bugs item #478887, was opened at 2001-11-06 14:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478887&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: imaplib patch Initial Comment: The comment in the patch says it all. I am using Linux (various distros/versions) with Jython 2.1a3. *** /usr/lib/python2.1/imaplib.py Sat Oct 20 10:48:35 2001 --- lib/modules/imaplib.py Tue Nov 6 16:59:15 2001 *************** *** 198,204 **** """Setup 'self.sock' and 'self.file'.""" self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect((self.host, self.port)) ! self.file = self.sock.makefile('r') def recent(self): --- 198,206 ---- """Setup 'self.sock' and 'self.file'.""" self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) self.sock.connect((self.host, self.port)) ! # use binary mode -- with jython (java) not specifying binary mode ! # causes a CRLF->LF translation and screws up the data later on ! self.file = self.sock.makefile('rb') def recent(self): ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:01 Message: Logged In: YES user_id=21627 This was fixed in imaplib 1.35, in response to bug report #469910. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478887&group_id=5470 From noreply@sourceforge.net Wed Nov 7 22:02:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 14:02:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Wed Nov 7 22:11:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 14:11:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Wed Nov 7 22:13:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 14:13:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-471942 ] python2.1.1 SEGV in GC on Solaris 2.7 Message-ID: Bugs item #471942, was opened at 2001-10-16 19:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Anthony Baxter (anthonybaxter) Assigned to: Neil Schemenauer (nascheme) Summary: python2.1.1 SEGV in GC on Solaris 2.7 Initial Comment: I've got a Zope installation where python2.1.1 is segfaulting on Solaris2.7 - it's running a largish ZEO server. The tail of the gdb output is: #128 0x26164 in PyEval_CallObjectWithKeywords () #129 0x264c0 in PyEval_CallObjectWithKeywords () #130 0x26140 in PyEval_CallObjectWithKeywords () #131 0x25fc0 in PyEval_CallObjectWithKeywords () #132 0x517bc in PyInstance_New () #133 0x261a4 in PyEval_CallObjectWithKeywords () #134 0x25fc0 in PyEval_CallObjectWithKeywords () #135 0x42c90 in initgc () It's built with $ gcc -v Reading specs from /opt/local/lib/gcc-lib/sparc-sun-solaris2.7/2.95.2/specs gcc version 2.95.2 19991024 (release) which is a bit old. I'm going to rebuild with gcc3.0 and also try turning off the GC. Unfortunately I can't get this to happen on a smaller test system - it's only under load that it plows into the ground. I'll also leave symbols in this time... :/ ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:13 Message: Logged In: YES user_id=21627 Any news on this report: did the problem disappear after backporting changes? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 04:32 Message: Logged In: YES user_id=21627 Looking at the changes since 2.1.1, I can see one bugfix that might explain strange behaviour, namely ceval.c 2.277. If you want to try it out, you can get the change here http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Python/ceval.c.diff?r1=2.276&r2=2.277 ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:59 Message: Logged In: YES user_id=29957 What's the appropriate way to ask this? drop a line to python-dev? purify's showing a UMR in strop_expandtabs (from memory - don't have it on screen right now) when python starts up. Yeah, the realloc bug concerns me - I have to wonder if something's a little bit/lot screwy. I just don't know where. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:46 Message: Logged In: YES user_id=21627 This is something to ask the pythonlabs folks; I believe Python has been purified once in a while - perhaps even with Zope extensions. I'd still suspect a bug in the Zope extensions instead of a Python bug, though: if malloc crashes, chances are high that somebody was overwriting free memory. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-18 03:31 Message: Logged In: YES user_id=29957 It's not a GC object. I'm positive all the extension objects are correct - I just recompiled, without the 1.5/2.0 headers around. It's a different pointer each time round, unfortunately. It also takes anything from 5 minutes to 2 hours to reproduce. I've got about 4 copies of it running now, and I've got a bunch of different core files. I've grabbed purify and an eval license, and I'm feeding it the binary. The printf approach is probably not going to work - these are busy busy Zope servers. Instead, my plan, assuming that purify doesn't immediately spot a problem, is to change the code so that if it gets a dud GC object, it will just bust it out of the tree and let it leak, and print a message saying so. Then I can quit the program, and purify will tell me 'hey, you leaked!' and also tell me where it was allocated. More concerning, about half the segfaults are not from the GC at all, but from realloc in PyFrame_New (line 161 of frameobject). These are the only two I'm getting - it's split 50-50 amongst the 10 coredumps I have now. I'm not sure whether to open a seperate bug for this. Has python2.1.1 been purified? With Zope and zope's extensions? Wow - it's amazing how this SF bug thing is so painful for conversations :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-18 03:11 Message: Logged In: YES user_id=21627 There are two options: a) the object isn't really a GC object, i.e. has no GC header. In gdb, you can try to cast gc to PyObject*, then see if the resulting pointer has a better ob_type (this is unlikely, though, since the logic entering the object was already using fromgc/togc) b) somebody has cleared the ob_type field. Can you guarantee that all extension modules have been compiled with the 2.1.1 header files? Is the problem repeatable in the sense that gc will have the same pointer value on each crash? If so, it is relatively easy to track down: just set a gdb change watchpoint on the address on the ob_type field of that address (note that setting watchpoints is not possible until there is really mapped memory on that address). If you can't analyse it through change breakpoints, I recommend to annotate the interpreter in the following way: in pyobject_init, put a printf that prints the address and the tp_name of the type. In subtract_refs, if the ob_type slot is null, print the address of the object and abort. Then analyse the log to see whether a object really has been allocated on that address, and what its type was (make sure you consider the possibility that address are off by the delta that FROM_GC adds). ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 21:58 Message: Logged In: YES user_id=29957 Ok, I have an intact core file, and a matching binary, no optimisations, nothing. This crash is showing the crash at line 166 of gcmodule.c traverse = PyObject_FROM_GC(gc)->ob_type->tp_traverse; PyObject_FROM_GC(gc)->ob_type in this case is $24 = {ob_refcnt = 1, ob_type = 0x0} To check my logic, I checked gc_next and gc_prev using the same GDB magic, and they correctly show up as a tuple and an instance method. Some fiddling around seems to rule out stack space as the problem, as well. We're going to try and see if purify helps here, but the problem looks to be a junk object - I have no idea how to track this down further. Help? Would taking the horrible horrible hack of removing the object from the gc linked list if ob_type is null help? Well, it'd stop the crashes, anyway. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-17 13:44 Message: Logged In: YES user_id=21627 It would be interesting what the value of "gc" is at the time of the crash. It looks like you got an object that claims to support GC but has a null tp_traverse. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2001-10-17 06:08 Message: Logged In: YES user_id=29957 I'm a doofus who read the gdb trace from the wrong end - too much python lately :) Nonetheless, the other end of the trace failed in gc as well - and building without GC enabled worked. Here's the trace with debugging enabled: #0 0xff00 in ?? () #1 0x402f0 in collect (young=0x9b538, old=0x9b544) at ./Modules/gcmodule.c:379 #2 0x405a8 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x40624 in _PyGC_Insert (op=0xbc1f24) at ./Modules/gcmodule.c:507 #4 0x5a224 in PyList_New (size=0) at Objects/listobject.c:61 #5 0x21bc8 in eval_code2 (co=0x1cb370, globals=0x21bc0, locals=0x67, args=0x0, argcount=1, kws=0xf89b24, kwcount=0, defs=0x0, defcount=0, closure=0xbc1f24) at Python/ceval.c:1741 Next trick is to rebuild without any optimisation (sigh) as I suspect that it's inlined subtract_refs(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471942&group_id=5470 From noreply@sourceforge.net Wed Nov 7 22:19:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 14:19:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-478006 ] memory leaks in Modules & threads Message-ID: Bugs item #478006, was opened at 2001-11-04 07:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leaks in Modules & threads Initial Comment: the attached patch corrects memory leaks found when running the regression tests. Files: Modules/_hotshot.c Modules/sunaudiodev.c Python/thread_pthread.h ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:19 Message: Logged In: YES user_id=21627 I believe all three chunks (_hotshot.c, sunaudiodev.c, thread_pthread.h) are correct. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 15:07 Message: Logged In: YES user_id=31435 Neal, if you do more of these, could you please limit them to one module per patch? Not all the suggested fixes have made sense, and the report gets to be a mess when only part of a patch can be applied. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 From noreply@sourceforge.net Wed Nov 7 22:22:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 14:22:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-477700 ] memory leaks in gdbm & curses Message-ID: Bugs item #477700, was opened at 2001-11-02 18:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Martin v. Löwis (loewis) Summary: memory leaks in gdbm & curses Initial Comment: Purify reports memory leaks in gdbm & curses modules. The attached patch fixes these two small leaks. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:22 Message: Logged In: YES user_id=21627 The patch looks fine to me. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-04 14:49 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 From noreply@sourceforge.net Wed Nov 7 23:54:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 07 Nov 2001 15:54:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Thu Nov 8 08:03:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 00:03:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-479469 ] File copy fails on True64 AFS file syste Message-ID: Bugs item #479469, was opened at 2001-11-08 00:03 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479469&group_id=5470 Category: Distutils Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Konrad Hinsen (hinsen) Assigned to: A.M. Kuchling (akuchling) Summary: File copy fails on True64 AFS file syste Initial Comment: The following report comes from an MMTK user who has a peculiar installation problem which is probably caused by some Distutils bug/feature: ------------------------------------------------ As an aside, I'm having problems with the installation of MMTK itself on Alpha/Tru64. I keep getting these: copying MMTK/Database/Groups/aspartic_acid_uni2 -> /afs/bi/v/@sys/languages/python/latest/lib/python2.0/site-packages/MMTK/Database/Groups error: /afs/bi/v/@sys/languages/python/latest/lib/python2.0/site-packages/MMTK/Database/Groups/aspartic_acid_uni2: Not owner The file is copied, but the installation process halts there. I am able to proceed by rerunning the python setup.py install; it notices that that file got there OK, copies the next one, and halts again. As you notice, we have the AFS file system (whose chmod/chown semantics are quite different from regular UNIX). This is not a problem on Linux, HP, SGI or Sun, only on Alpha. Also, it does not matter whether I have my normal account or the administrator account on AFS. What is the installation program trying to do and is it altogether necessary? ------------------------------------------------- The setup.py script is attached. The file mentioned above is one of the data files that end up in the data_files variable. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479469&group_id=5470 From noreply@sourceforge.net Thu Nov 8 13:48:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 05:48:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-479568 ] xxsubtype builtin Message-ID: Bugs item #479568, was opened at 2001-11-08 05:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479568&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: xxsubtype builtin Initial Comment: Python 2.2b1+ (#1, Nov 6 2001, 21:52:55) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "help", "copyright", "credits" or "license" for more information. >>> import sys sys.builtin_module_names ('__builtin__', '__main__', '_sre', '_symtable', 'exceptions', 'gc', 'imp', 'marshal', 'new', 'posix', 'signal', 'sys', 'thread', 'xxsubtype') >>> Should xxsubtype be a builtin module? The line mentioning it in Setup.dist is uncommented by default. Maybe it's there just for testing purposes? Thanks! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479568&group_id=5470 From noreply@sourceforge.net Thu Nov 8 13:55:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 05:55:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-479571 ] --disable-unicode not working Message-ID: Bugs item #479571, was opened at 2001-11-08 05:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479571&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: --disable-unicode not working Initial Comment: In Python/sysmodule.c, sys_getdefaultencoding() function should be "protected" by Py_USING_UNICODE definition. Thanks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479571&group_id=5470 From noreply@sourceforge.net Thu Nov 8 14:10:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 06:10:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 16:47:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 08:47:38 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 >Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 17:09:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 09:09:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 17:11:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 09:11:21 -0800 Subject: [Python-bugs-list] [ python-Bugs-479571 ] --disable-unicode not working Message-ID: Bugs item #479571, was opened at 2001-11-08 05:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479571&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) >Assigned to: Martin v. Löwis (loewis) Summary: --disable-unicode not working Initial Comment: In Python/sysmodule.c, sys_getdefaultencoding() function should be "protected" by Py_USING_UNICODE definition. Thanks. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479571&group_id=5470 From noreply@sourceforge.net Thu Nov 8 17:22:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 09:22:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 17:33:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 09:33:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 17:37:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 09:37:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Documentation >Group: Feature Request Status: Open Resolution: None >Priority: 4 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Thu Nov 8 17:48:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 09:48:33 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 18:53:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 10:53:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-479698 ] __coerce__ not working with new classes Message-ID: Bugs item #479698, was opened at 2001-11-08 10:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479698&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __coerce__ not working with new classes Initial Comment: I converted a numeric-like class from the "old-style" Python class to the "new-style" Python class by inheriting from "object". Numerical operations like "__mul__" and "__div__" failed with the "new-style" class. After tracking the problem, I found that the "__coerce__" special method was called when performing mixed-type arithmetic, as expected, with the "old-style" class, but not with the "new-style" class. The arithmetic operations failed because the expected type conversion was not performed with the mixed-type arithmetic. As best I can tell, without examining the Python source code, the following is taking place: 1. The documentation for the "__coerce__" special method states that it is invoked in the case of mixed-type arithmetic only if at least one of the arguments in an arithmetic operation is a class instance. 2. An object created with an "old-style" class is of type "instance", as verified by the "type" function, i.e. 3. An object created with a "new-style" class is of a different type, as also verified by the "type function, i.e. Evidently, because of this change in type, the "__coerce__" function is not longer invoked for "new-style" Python classes. Gary H. Loechelt ON Semiconductor ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479698&group_id=5470 From noreply@sourceforge.net Thu Nov 8 19:22:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 11:22:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 19:49:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 11:49:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 20:03:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 12:03:07 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Thu Nov 8 23:28:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 15:28:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Fri Nov 9 01:09:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 08 Nov 2001 17:09:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Fri Nov 9 09:21:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 01:21:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 4 Submitted By: Tim Peters (tim_one) Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Fri Nov 9 12:41:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 04:41:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-479967 ] Python/C API manual has no index section Message-ID: Bugs item #479967, was opened at 2001-11-09 04:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Thomas Heller (theller) Assigned to: Nobody/Anonymous (nobody) Summary: Python/C API manual has no index section Initial Comment: In the development docs, the Python/C API manual the index section is missing. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 From noreply@sourceforge.net Fri Nov 9 12:42:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 04:42:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-479967 ] Python/C API manual has no index section Message-ID: Bugs item #479967, was opened at 2001-11-09 04:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 >Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Thomas Heller (theller) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Python/C API manual has no index section Initial Comment: In the development docs, the Python/C API manual the index section is missing. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 From noreply@sourceforge.net Fri Nov 9 13:02:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 05:02:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-479981 ] socket failed then compiled with treads Message-ID: Bugs item #479981, was opened at 2001-11-09 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 Category: Threads Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket failed then compiled with treads Initial Comment: Operating System HP-UX 11.0 (64 Bit) I tried to compile Python 2.1.1 this thread support. Doing so, the tests test_socket and test_asynchat both failed with "(9, 'Bad file number')" The test_socket succeds if I compiled python without threads, so I think it's an thread-problem. The thread-test istself succeeds. I think also, that python is correctly linked against libpthread. The same Problem occurs this Python 2.2.b1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 From noreply@sourceforge.net Fri Nov 9 14:44:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 06:44:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-479967 ] Python/C API manual has no index section Message-ID: Bugs item #479967, was opened at 2001-11-09 04:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None >Priority: 7 Submitted By: Thomas Heller (theller) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Python/C API manual has no index section Initial Comment: In the development docs, the Python/C API manual the index section is missing. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-09 06:43 Message: Logged In: YES user_id=3066 Raised the severity of this -- this *must* be fixed! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 From noreply@sourceforge.net Fri Nov 9 15:16:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 07:16:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Fri Nov 9 15:19:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 07:19:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-476593 ] Bad mode for opening file gives bad msg. Message-ID: Bugs item #476593, was opened at 2001-10-30 17:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476593&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Gregory H. Ball (greg_ball) >Assigned to: Jeremy Hylton (jhylton) Summary: Bad mode for opening file gives bad msg. Initial Comment: Calling the open (or file) builtin with an incorrect mode (for example, 'bw' or 's') causes an exception which claims that the filename is invalid. >>> open('tmp','s') Traceback (innermost last): File "", line 1, in ? IOError: [Errno 22] Invalid argument: 'tmp' (double spaced in hopes of preserving line breaks) It comes down to this call: PyErr_SetFromErrnoWithFilename(PyExc_IOError, name); which assumes that the filename is relevant to any IOError that pops up. man fopen on linux says: EINVAL The mode provided to fopen, fdopen, or freopen was invalid. so it seems possible to catch this case, but not without making the code more complex. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476593&group_id=5470 From noreply@sourceforge.net Fri Nov 9 15:19:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 07:19:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-478006 ] memory leaks in Modules & threads Message-ID: Bugs item #478006, was opened at 2001-11-04 07:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: memory leaks in Modules & threads Initial Comment: the attached patch corrects memory leaks found when running the regression tests. Files: Modules/_hotshot.c Modules/sunaudiodev.c Python/thread_pthread.h ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:19 Message: Logged In: YES user_id=21627 I believe all three chunks (_hotshot.c, sunaudiodev.c, thread_pthread.h) are correct. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 15:07 Message: Logged In: YES user_id=31435 Neal, if you do more of these, could you please limit them to one module per patch? Not all the suggested fixes have made sense, and the report gets to be a mess when only part of a patch can be applied. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 From noreply@sourceforge.net Fri Nov 9 15:37:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 07:37:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-479186 ] compiler generates bad code for "del" Message-ID: Bugs item #479186, was opened at 2001-11-07 09:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479186&group_id=5470 Category: Python Library Group: None Status: Open >Resolution: Accepted Priority: 5 Submitted By: Neil Schemenauer (nascheme) Assigned to: Jeremy Hylton (jhylton) >Summary: compiler generates bad code for "del" Initial Comment: The compiler generates bad code from the following statement: del a[0], a[1] Here is the AST: Stmt([AssTuple([Subscript(Name('a'), 'OP_DELETE', [Const(0)]), Subscript(Name('a'), 'OP_DELETE', [Const(1)])])]) The bytecode generated is: 0 SET_LINENO 0 3 UNPACK_SEQUENCE 2 BAD ------^^^^^^^^^^^^^^^^^^^^^^^^^^ 6 SET_LINENO 2 9 LOAD_NAME 0 (a) 12 LOAD_CONST 1 (0) 15 DELETE_SUBSCR 16 LOAD_NAME 0 (a) 19 LOAD_CONST 2 (1) 22 DELETE_SUBSCR The problem, AFAICT, is that findOp does not find OP_DELETE. Adding a visitSubscript method to OpFinder seems to be the fix. A small patch is attached. This is a 2.1.2 bugfix candidate, IMO. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479186&group_id=5470 From noreply@sourceforge.net Fri Nov 9 15:51:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 07:51:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Fri Nov 9 16:02:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 08:02:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-478006 ] memory leaks in Modules & threads Message-ID: Bugs item #478006, was opened at 2001-11-04 07:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: memory leaks in Modules & threads Initial Comment: the attached patch corrects memory leaks found when running the regression tests. Files: Modules/_hotshot.c Modules/sunaudiodev.c Python/thread_pthread.h ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-09 08:02 Message: Logged In: YES user_id=3066 Committed, with a (minor) change to the _hotshot.c patch (added the free() immediately before variable was re-used). Modules/_hotshot.c 1.9 Modules/sunaudiodev.c 1.25 Python/thread_pthread.h 2.36 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:19 Message: Logged In: YES user_id=21627 I believe all three chunks (_hotshot.c, sunaudiodev.c, thread_pthread.h) are correct. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 15:07 Message: Logged In: YES user_id=31435 Neal, if you do more of these, could you please limit them to one module per patch? Not all the suggested fixes have made sense, and the report gets to be a mess when only part of a patch can be applied. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478006&group_id=5470 From noreply@sourceforge.net Fri Nov 9 16:16:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 08:16:41 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Fri Nov 9 16:18:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 08:18:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-476593 ] Bad mode for opening file gives bad msg. Message-ID: Bugs item #476593, was opened at 2001-10-30 17:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476593&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Gregory H. Ball (greg_ball) Assigned to: Jeremy Hylton (jhylton) Summary: Bad mode for opening file gives bad msg. Initial Comment: Calling the open (or file) builtin with an incorrect mode (for example, 'bw' or 's') causes an exception which claims that the filename is invalid. >>> open('tmp','s') Traceback (innermost last): File "", line 1, in ? IOError: [Errno 22] Invalid argument: 'tmp' (double spaced in hopes of preserving line breaks) It comes down to this call: PyErr_SetFromErrnoWithFilename(PyExc_IOError, name); which assumes that the filename is relevant to any IOError that pops up. man fopen on linux says: EINVAL The mode provided to fopen, fdopen, or freopen was invalid. so it seems possible to catch this case, but not without making the code more complex. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-09 08:18 Message: Logged In: YES user_id=31392 Fixed in rev 2.137 of fileobject.c. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476593&group_id=5470 From noreply@sourceforge.net Fri Nov 9 16:24:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 08:24:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-479186 ] compiler generates bad code for "del" Message-ID: Bugs item #479186, was opened at 2001-11-07 09:14 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479186&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Neil Schemenauer (nascheme) Assigned to: Jeremy Hylton (jhylton) >Summary: compiler generates bad code for "del" Initial Comment: The compiler generates bad code from the following statement: del a[0], a[1] Here is the AST: Stmt([AssTuple([Subscript(Name('a'), 'OP_DELETE', [Const(0)]), Subscript(Name('a'), 'OP_DELETE', [Const(1)])])]) The bytecode generated is: 0 SET_LINENO 0 3 UNPACK_SEQUENCE 2 BAD ------^^^^^^^^^^^^^^^^^^^^^^^^^^ 6 SET_LINENO 2 9 LOAD_NAME 0 (a) 12 LOAD_CONST 1 (0) 15 DELETE_SUBSCR 16 LOAD_NAME 0 (a) 19 LOAD_CONST 2 (1) 22 DELETE_SUBSCR The problem, AFAICT, is that findOp does not find OP_DELETE. Adding a visitSubscript method to OpFinder seems to be the fix. A small patch is attached. This is a 2.1.2 bugfix candidate, IMO. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-09 08:24 Message: Logged In: YES user_id=31392 Fixed in rev 1.57 of pycodegen.py. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479186&group_id=5470 From noreply@sourceforge.net Fri Nov 9 16:48:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 08:48:38 -0800 Subject: [Python-bugs-list] [ python-Bugs-451295 ] HTTPS bugs in urllib2 Message-ID: Bugs item #451295, was opened at 2001-08-15 12:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451295&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: HTTPS bugs in urllib2 Initial Comment: 1. In AbstractBasicAuthHandler.retry_http_basic_auth()- if an HTTP error, other than 401, happens, authentication is disabled because self.__current_realm is not set to None 2. In HTTPRedirectHandler.http_error_302()- if an authenticated file is redirected, the old request header is discarded and, therefore, so is the authentication information. If there is a new challenge, HTTP error 401 results, self.__current_realm is still set and authentication is thereafter disabled. I discovered these bugs while testing with a secure web site. I made the following fixes (standard Unix diff format) and can be reached via email at bcox@semio.com: bug #1- 631,633c649,660 < resp = self.parent.open(req) < self.__current_realm = None < return resp --- > try: > resp = self.parent.open(req) > self.__current_realm = None > return resp > # prevent other HTTP errors from disabling authentication > except HTTPError, e: > if e.code != 401: > self.__current_realm = None > raise > except (URLError, socket.error), e: > self.__current_realm = None > raise bug #2- 447c460,465 < new = Request(newurl, req.get_data()) --- > > # forgetting about the current state is not a good idea. > # If this is an authenticated URL, then discarding the state > # (i.e. the headers) will disable further authentication > > new = Request(newurl, req.get_data(), req.headers) ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-09 08:48 Message: Logged In: YES user_id=31392 Fixed in rev 1.24 of urllib2.py based on patch from 468948. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451295&group_id=5470 From noreply@sourceforge.net Fri Nov 9 16:48:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 08:48:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-451295 ] HTTPS bugs in urllib2 Message-ID: Bugs item #451295, was opened at 2001-08-15 12:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451295&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jeremy Hylton (jhylton) Summary: HTTPS bugs in urllib2 Initial Comment: 1. In AbstractBasicAuthHandler.retry_http_basic_auth()- if an HTTP error, other than 401, happens, authentication is disabled because self.__current_realm is not set to None 2. In HTTPRedirectHandler.http_error_302()- if an authenticated file is redirected, the old request header is discarded and, therefore, so is the authentication information. If there is a new challenge, HTTP error 401 results, self.__current_realm is still set and authentication is thereafter disabled. I discovered these bugs while testing with a secure web site. I made the following fixes (standard Unix diff format) and can be reached via email at bcox@semio.com: bug #1- 631,633c649,660 < resp = self.parent.open(req) < self.__current_realm = None < return resp --- > try: > resp = self.parent.open(req) > self.__current_realm = None > return resp > # prevent other HTTP errors from disabling authentication > except HTTPError, e: > if e.code != 401: > self.__current_realm = None > raise > except (URLError, socket.error), e: > self.__current_realm = None > raise bug #2- 447c460,465 < new = Request(newurl, req.get_data()) --- > > # forgetting about the current state is not a good idea. > # If this is an authenticated URL, then discarding the state > # (i.e. the headers) will disable further authentication > > new = Request(newurl, req.get_data(), req.headers) ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-09 08:48 Message: Logged In: YES user_id=31392 Fixed in rev 1.24 of urllib2.py based on patch from 468948. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=451295&group_id=5470 From noreply@sourceforge.net Fri Nov 9 16:49:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 08:49:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-468948 ] urllib2, basic authentication, & 302 Message-ID: Bugs item #468948, was opened at 2001-10-07 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Jeffrey C. Ollie (jcollie) Assigned to: Jeremy Hylton (jhylton) Summary: urllib2, basic authentication, & 302 Initial Comment: I've found a bug in how urllib2 handles authentication. The crux of the problem is that the AbstractBasicAuthHandler.__current_realm is basically a global variable. I discovered the bug because I needed to use HTTP basic authentication and the HTTP If-Modified-Since header. Since the HTTP server returns a 302 error if the requested URL has not changed the line of code that resets AbstractBasicAuthHandler.__current_realm back to None never gets executed because an exception will be raised when the retrieval is retried. I suspect that this bug would also cause problems in multi-threaded code. The digest authentication appears to have similar problems. The solution that I found is to get rid of the __current_realm attribute and prevent infinite retries by checking for the presence of an Authenticate: header in the request object that exactly matches the Authenticate: header that would be added. The bug exists in 2.1.1, 2.2a4 and the current CVS. Patch attached. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-09 08:49 Message: Logged In: YES user_id=31392 Fixed in rev. 1.24 of urllib2.py, including changes to digest authentication. ---------------------------------------------------------------------- Comment By: Jeffrey C. Ollie (jcollie) Date: 2001-10-09 19:54 Message: Logged In: YES user_id=37310 I haven't tried the patch yet but I think that it would fix the bug in the case of a single-threaded program. However, with a multi-threaded program I think that my patch for this problem is superior because it doesn't rely on a property shared by all of the threads that use the same opener object. Of course, my patch assumes that a request object is used only by one thread but I think that that's a safer assumption. ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-10-09 19:17 Message: Logged In: YES user_id=31392 Does the patch in bug #451295 fix your problem? ---------------------------------------------------------------------- Comment By: Jeffrey C. Ollie (jcollie) Date: 2001-10-09 10:21 Message: Logged In: YES user_id=37310 Oops, that patch that I attached is reversed, use -R! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=468948&group_id=5470 From noreply@sourceforge.net Fri Nov 9 17:12:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 09:12:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-480096 ] Assign to __debug__ still allowed Message-ID: Bugs item #480096, was opened at 2001-11-09 09:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480096&group_id=5470 Category: Parser/Compiler Group: Python 2.2 Status: Open Resolution: None Priority: 4 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: Assign to __debug__ still allowed Initial Comment: >From the 2.1c1 NEWS file: """ - After a public outcry, assignment to __debug__ is no longer illegal; instead, a warning is issued. It will become illegal in 2.2. """ That hasn't yet happened. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480096&group_id=5470 From noreply@sourceforge.net Fri Nov 9 18:17:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 10:17:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Fri Nov 9 19:30:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 11:30:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Fri Nov 9 21:00:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 13:00:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-479571 ] --disable-unicode not working Message-ID: Bugs item #479571, was opened at 2001-11-08 05:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479571&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Martin v. Löwis (loewis) Summary: --disable-unicode not working Initial Comment: In Python/sysmodule.c, sys_getdefaultencoding() function should be "protected" by Py_USING_UNICODE definition. Thanks. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 13:00 Message: Logged In: YES user_id=21627 Thanks for the report; this is fixed in sysmodule.c 2.95. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479571&group_id=5470 From noreply@sourceforge.net Fri Nov 9 21:10:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 13:10:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-480188 ] printing unicode objects Message-ID: Bugs item #480188, was opened at 2001-11-09 13:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: printing unicode objects Initial Comment: """ In a print statement using a trailing comma (to suppress printing a newline) printing a unicode object that contains a trailing newline sometimes prints an extra space. If I have a unicode object, u, that contains a trailing newline, then... print str(u), # behaves as expected print u.encode(), # behaves as expected print u, # the newline isn't recognized # and the next print # inserts a space It appears that the implicit form of encoding the unicode object doesn't quite work right with print. The following code illustrates my problem. My output follows. """ import sys print sys.version s = 'This is a test\n' u = unicode(s) print print s, print s, print s, print print str(u), print str(u), print str(u), print print u.encode(), print u.encode(), print u.encode(), print print u, print u, print u, """ My output: 2.1.1 (#1, Aug 28 2001, 19:51:39) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-85)] This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test """ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 From noreply@sourceforge.net Fri Nov 9 22:33:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 14:33:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-480215 ] softspace confused in nested print Message-ID: Bugs item #480215, was opened at 2001-11-09 14:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480215&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: softspace confused in nested print Initial Comment: Adapted from a c.l.py report: class C: . def __str__(self): . print "a" . return "b" print C() The output is " a\nb\n" -- note the surprising leading space. This is because PRINT_ITEM forces softspace to 1 *before* converting the output item (so softspace is 1 when we get into __str__, despite that nothing yet has been written). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480215&group_id=5470 From noreply@sourceforge.net Fri Nov 9 22:57:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 14:57:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-480188 ] printing unicode objects Message-ID: Bugs item #480188, was opened at 2001-11-09 13:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: printing unicode objects Initial Comment: """ In a print statement using a trailing comma (to suppress printing a newline) printing a unicode object that contains a trailing newline sometimes prints an extra space. If I have a unicode object, u, that contains a trailing newline, then... print str(u), # behaves as expected print u.encode(), # behaves as expected print u, # the newline isn't recognized # and the next print # inserts a space It appears that the implicit form of encoding the unicode object doesn't quite work right with print. The following code illustrates my problem. My output follows. """ import sys print sys.version s = 'This is a test\n' u = unicode(s) print print s, print s, print s, print print str(u), print str(u), print str(u), print print u.encode(), print u.encode(), print u.encode(), print print u, print u, print u, """ My output: 2.1.1 (#1, Aug 28 2001, 19:51:39) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-85)] This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test """ ---------------------------------------------------------------------- Comment By: Daniel Popowich (dpopowich) Date: 2001-11-09 14:57 Message: Logged In: YES user_id=372379 When I submitted this but I didn't realize the form was going to eat my formatting. The last few lines of the output has leading spaces. Run the sample code to see what I mean. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 From noreply@sourceforge.net Fri Nov 9 23:51:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 15:51:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-479981 ] socket failed then compiled with treads Message-ID: Bugs item #479981, was opened at 2001-11-09 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 Category: Threads Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket failed then compiled with treads Initial Comment: Operating System HP-UX 11.0 (64 Bit) I tried to compile Python 2.1.1 this thread support. Doing so, the tests test_socket and test_asynchat both failed with "(9, 'Bad file number')" The test_socket succeds if I compiled python without threads, so I think it's an thread-problem. The thread-test istself succeeds. I think also, that python is correctly linked against libpthread. The same Problem occurs this Python 2.2.b1. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 15:51 Message: Logged In: YES user_id=21627 What compiler are you using? Can you please report the complete compiler line used to link together the python binary? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 From noreply@sourceforge.net Sat Nov 10 00:13:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 09 Nov 2001 16:13:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-479981 ] socket failed then compiled with treads Message-ID: Bugs item #479981, was opened at 2001-11-09 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 Category: Threads Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket failed then compiled with treads Initial Comment: Operating System HP-UX 11.0 (64 Bit) I tried to compile Python 2.1.1 this thread support. Doing so, the tests test_socket and test_asynchat both failed with "(9, 'Bad file number')" The test_socket succeds if I compiled python without threads, so I think it's an thread-problem. The thread-test istself succeeds. I think also, that python is correctly linked against libpthread. The same Problem occurs this Python 2.2.b1. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 16:13 Message: Logged In: YES user_id=21627 Can you please also check whether the patches PHKL_23842 (or equivalent) and/or PHKL_20942. The patch titles are PHKL_23842 s700_800 11.00 signal,threads,spinlock,scheduler,IDS,q3p PHKL_20942 s700_800 11.00 signal, nanosleep, spinlock, scheduler It appears that HP-UX has a number of bugs where signals, forking, and other aspects of the operating functionality stop working in the presence of threads. If you don't have these patches, it would be good if you could try to install them, and report back whether it changes anything. Notice that HP apparently has a number of patches with these titles; they seem to be intended for different machines. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 15:51 Message: Logged In: YES user_id=21627 What compiler are you using? Can you please report the complete compiler line used to link together the python binary? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 From noreply@sourceforge.net Sat Nov 10 09:20:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 01:20:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-480325 ] Tkinter crash on Win2000, OK on UNIX Message-ID: Bugs item #480325, was opened at 2001-11-10 01:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 Category: Tkinter Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: John W Lewis (lewisjwl) Assigned to: Nobody/Anonymous (nobody) Summary: Tkinter crash on Win2000, OK on UNIX Initial Comment: Large Python/Tkinter application.... 2GB virtual, 3600 images, 3600 graphic objects, 3600x3600 arrays 3200x1600 dual monitor graphics Runs OK under 2.1.1 Python and Solaris8 Crashes in TK on Windows2000 at 3200 images DELL 530 workstation with 4GB RAM Windows 2000 SR1 2.1.1 Python How do I go about isolating this problem? Thanks John Lewis ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 From noreply@sourceforge.net Sat Nov 10 10:13:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 02:13:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Sat Nov 10 11:21:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 03:21:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-480337 ] Tut: Dict used before dicts explained Message-ID: Bugs item #480337, was opened at 2001-11-10 03:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480337&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Andrew Bennetts (spiv) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Tut: Dict used before dicts explained Initial Comment: This was pointed out to me by someone on #python, who was rather confused by this... In Section 5.1.4 of the Tutorial it has an example of a list comprehension: """ >>> [{x: x**2} for x in vec] [{2: 4}, {4: 16}, {6: 36}] """ But the "{x: x**2}" is confusing, because dictionaries haven't been explained yet (that's Section 5.4). This example either needs to be removed or modified. It is a good example of the sort of expression you can use in a list comprehension, so I'd recommend adding an comment referring the reader to Section 5.4, rather than removing it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480337&group_id=5470 From noreply@sourceforge.net Sat Nov 10 14:07:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 06:07:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-480325 ] Tkinter crash on Win2000, OK on UNIX Message-ID: Bugs item #480325, was opened at 2001-11-10 01:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 Category: Tkinter Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: John W Lewis (lewisjwl) Assigned to: Nobody/Anonymous (nobody) Summary: Tkinter crash on Win2000, OK on UNIX Initial Comment: Large Python/Tkinter application.... 2GB virtual, 3600 images, 3600 graphic objects, 3600x3600 arrays 3200x1600 dual monitor graphics Runs OK under 2.1.1 Python and Solaris8 Crashes in TK on Windows2000 at 3200 images DELL 530 workstation with 4GB RAM Windows 2000 SR1 2.1.1 Python How do I go about isolating this problem? Thanks John Lewis ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 06:07 Message: Logged In: YES user_id=21627 I recommend to attach to the process with a debugger (e.g. VC++). If you manage to do this, please report the stack trace at the moment of the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 From noreply@sourceforge.net Sat Nov 10 17:02:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 09:02:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-480384 ] UMR in curses module Message-ID: Bugs item #480384, was opened at 2001-11-10 09:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: UMR in curses module Initial Comment: passing &char as a null terminated string results in uninitialized memory reads change char to char[2] and null-terminate the array ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 From noreply@sourceforge.net Sat Nov 10 17:37:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 09:37:33 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 >Category: Extension Modules >Group: None Status: Open Resolution: None >Priority: 7 Submitted By: Tim Peters (tim_one) >Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Sat Nov 10 19:42:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 11:42:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Sat Nov 10 22:07:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 14:07:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Sun Nov 11 00:22:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 16:22:04 -0800 Subject: [Python-bugs-list] [ python-Bugs-480480 ] pydoc #!/build-directory/python Message-ID: Bugs item #480480, was opened at 2001-11-10 16:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480480&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Brian Zhou (bzhou) Assigned to: Nobody/Anonymous (nobody) Summary: pydoc #!/build-directory/python Initial Comment: Starting from source for example http://www.python.org/ftp/python/2.2/rpms/python2.2- 2.2b1-2.src.rpm "make install" copies and later packages a version of pydoc which has hard-coded #!/build-directory/python This pydoc will be installed on redhat 7.x. The unwanted dependency will also cause the binary RPM installation to fail on redhat 6.2. "make install" should be fixed to use "#!/usr/bin/env python" or simply pick-up Tools/scripts/pydoc. A quick workaround for rpm is to change the python-2.2.spec file: 162,163c162 < sed 's|#!/usr/bin/env python|#!/usr/bin/env python'%{binsuffix}'|' \ < pydoc.old >pydoc --- > sed 's|#!.*python$|#!/usr/bin/env python'% {binsuffix}'|' pydoc.old >pydoc ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480480&group_id=5470 From noreply@sourceforge.net Sun Nov 11 06:15:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 10 Nov 2001 22:15:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Sun Nov 11 09:15:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 01:15:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Sun Nov 11 10:27:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 02:27:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Sun Nov 11 10:37:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 02:37:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Mark Hammond (mhammond) Date: 2001-11-11 02:37 Message: Logged In: YES user_id=14198 *sigh* - forgot that the object layout changed :( Re malloc issues: they are indeed still a problem - but Python will not *force* you to do this. It used to be the case that an extension module would malloc an object, and whatever module did the final Py_DECREF (often the core) would perform the free(). I believe this is no longer true. It *is* still true with FILE objects and the tp_print slot IIRC. My point with the "_d" was that iff we can get things working so that the standard Python extensions are "safe", "_d" need not imply anything other than "the preferred extension for debug builds of the core" - and even that is only for b/w compat and convenience reasons. How has Linux survived so long? I assume Py_DEBUG builds are simply never made in the "wild". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Sun Nov 11 14:18:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 06:18:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:18 Message: Logged In: YES user_id=21627 On Linux, it is common to include debug information into every library and executable; in particular, it is common to install python with debugging information. This is possible since debugging information can be combined with optimization almost seamless in gcc (distributors often strip the debugging information, though). So you use Py_DEBUG *only* when you want to do the refcount debugging. You have to configure --with-pydebug for that, and you never do that in the wild. *If* you would install a pydebug binary, you would use a different --prefix, so that it would not interfere with the standard installation. Depending on which pyconfig.h extensions pick up, they would then be suitable for use with one or the other installation. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 02:37 Message: Logged In: YES user_id=14198 *sigh* - forgot that the object layout changed :( Re malloc issues: they are indeed still a problem - but Python will not *force* you to do this. It used to be the case that an extension module would malloc an object, and whatever module did the final Py_DECREF (often the core) would perform the free(). I believe this is no longer true. It *is* still true with FILE objects and the tp_print slot IIRC. My point with the "_d" was that iff we can get things working so that the standard Python extensions are "safe", "_d" need not imply anything other than "the preferred extension for debug builds of the core" - and even that is only for b/w compat and convenience reasons. How has Linux survived so long? I assume Py_DEBUG builds are simply never made in the "wild". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Sun Nov 11 14:21:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 06:21:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-480384 ] UMR in curses module Message-ID: Bugs item #480384, was opened at 2001-11-10 09:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: UMR in curses module Initial Comment: passing &char as a null terminated string results in uninitialized memory reads change char to char[2] and null-terminate the array ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:21 Message: Logged In: YES user_id=21627 Wouldn't it be better to use FromStringAndSize instead? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 From noreply@sourceforge.net Sun Nov 11 14:24:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 06:24:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-477700 ] memory leaks in gdbm & curses Message-ID: Bugs item #477700, was opened at 2001-11-02 18:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Martin v. Löwis (loewis) Summary: memory leaks in gdbm & curses Initial Comment: Purify reports memory leaks in gdbm & curses modules. The attached patch fixes these two small leaks. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:24 Message: Logged In: YES user_id=21627 Thanks for the patch, committed as _cursesmodule.c 2.59 and gdbmmodule.c 2.30. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:22 Message: Logged In: YES user_id=21627 The patch looks fine to me. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-04 14:49 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 From noreply@sourceforge.net Sun Nov 11 14:25:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 06:25:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-477700 ] memory leaks in gdbm & curses Message-ID: Bugs item #477700, was opened at 2001-11-02 18:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Martin v. Löwis (loewis) Summary: memory leaks in gdbm & curses Initial Comment: Purify reports memory leaks in gdbm & curses modules. The attached patch fixes these two small leaks. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:24 Message: Logged In: YES user_id=21627 Thanks for the patch, committed as _cursesmodule.c 2.59 and gdbmmodule.c 2.30. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:22 Message: Logged In: YES user_id=21627 The patch looks fine to me. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-04 14:49 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477700&group_id=5470 From noreply@sourceforge.net Sun Nov 11 14:30:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 06:30:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-480384 ] UMR in curses module Message-ID: Bugs item #480384, was opened at 2001-11-10 09:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: UMR in curses module Initial Comment: passing &char as a null terminated string results in uninitialized memory reads change char to char[2] and null-terminate the array ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2001-11-11 06:30 Message: Logged In: YES user_id=33168 Using FromStringAndSize also fixes the problem. I don't know which is better, but FromStringAndSize is a smaller change (just change the function call and add 1 as the second parameter) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:21 Message: Logged In: YES user_id=21627 Wouldn't it be better to use FromStringAndSize instead? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 From noreply@sourceforge.net Sun Nov 11 14:50:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 06:50:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-480384 ] UMR in curses module Message-ID: Bugs item #480384, was opened at 2001-11-10 09:02 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: UMR in curses module Initial Comment: passing &char as a null terminated string results in uninitialized memory reads change char to char[2] and null-terminate the array ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:50 Message: Logged In: YES user_id=21627 Thanks for the report. I've checked in the AndSize version as _cursesmodule.c 2.60, since it is the smaller change, and since it saves a strlen call (inside FromString). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2001-11-11 06:30 Message: Logged In: YES user_id=33168 Using FromStringAndSize also fixes the problem. I don't know which is better, but FromStringAndSize is a smaller change (just change the function call and add 1 as the second parameter) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:21 Message: Logged In: YES user_id=21627 Wouldn't it be better to use FromStringAndSize instead? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480384&group_id=5470 From noreply@sourceforge.net Sun Nov 11 15:02:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 07:02:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-480480 ] pydoc #!/build-directory/python Message-ID: Bugs item #480480, was opened at 2001-11-10 16:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480480&group_id=5470 >Category: Distutils Group: Python 2.2 Status: Open Resolution: None >Priority: 7 Submitted By: Brian Zhou (bzhou) Assigned to: Nobody/Anonymous (nobody) Summary: pydoc #!/build-directory/python Initial Comment: Starting from source for example http://www.python.org/ftp/python/2.2/rpms/python2.2- 2.2b1-2.src.rpm "make install" copies and later packages a version of pydoc which has hard-coded #!/build-directory/python This pydoc will be installed on redhat 7.x. The unwanted dependency will also cause the binary RPM installation to fail on redhat 6.2. "make install" should be fixed to use "#!/usr/bin/env python" or simply pick-up Tools/scripts/pydoc. A quick workaround for rpm is to change the python-2.2.spec file: 162,163c162 < sed 's|#!/usr/bin/env python|#!/usr/bin/env python'%{binsuffix}'|' \ < pydoc.old >pydoc --- > sed 's|#!.*python$|#!/usr/bin/env python'% {binsuffix}'|' pydoc.old >pydoc ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 07:02 Message: Logged In: YES user_id=21627 Putting in /usr/bin/env is certainly not the right solution. distutils was specifically designed to adjust the path of scripts, to make sure that the Python binary used at run-time is indeed the one used at installation time. Otherwise, the user may break pydoc by putting a different Python installation in front of the path. Instead, distutils should be fixed to assume that the Python interpreter will be in BINDIR. Recategorizing as distutils bug, and raising the priority. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480480&group_id=5470 From noreply@sourceforge.net Sun Nov 11 19:49:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 11:49:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-11 11:49 Message: Logged In: YES user_id=31435 Martin, the MS docs also say the linker generates a warning if you mix CRT types -- but it doesn't for mixing debug Python with expat or Tcl/Tk. I don't always believe everything in the MS docs . But, Mark, I do believe them when they say you're on thin ice, and they really don't document the ways in which mixing CRTs can be disastrous (they give three specific examples, "just a few" of "many [unspecified] ways to get into trouble"). Mark, I like _d fine! It's better than MS's plain "d" convention (which I've seen other projects mimic), and so long as there *is* a reason that mixing debug and release builds can be catastrophic, it's Good to make it painfully obvious that something potentially catastrophic is going on (note that Anonymous starting getting in trouble when he "disabled the use of the pragma in config.h" -- we spent most of this bug report trying to convince him the pragma wasn't there just to irritate him ). M and M, I don't *know* that expat.dll and Tcl/Tk work fine with a debug Python. All I've ever done with expat.dll then is run our test suite, and that simply doesn't crash; and I probably haven't run any Tcl/Tk scripts under a debug Python in years. Mark, I suspect debug Python builds are extremely rare on Unix, even among Python developers. For example, for more than a year, I believe I'm the only one who has found test- suite failures specific to the debug build. But "compile for debugging" to Unixoids usually means some combination of not defining NDEBUG, retaining symbol info, and/or cranking down optimization. The Unix libraries don't care about that stuff -- there's nothing there as Draconian as the MS-specific _DEBUG effect on libraries. Martin, It Hurts to tell people not to use _DEBUG on Windows: as a prime example, the MS _DEBUG malloc+free are indispensable debugging tools, and indeed have caught bugs in Python that escaped even efence. Users want that for their code too. I don't know where this leaves us. Probably right where we started, except we know better that we're lost . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:18 Message: Logged In: YES user_id=21627 On Linux, it is common to include debug information into every library and executable; in particular, it is common to install python with debugging information. This is possible since debugging information can be combined with optimization almost seamless in gcc (distributors often strip the debugging information, though). So you use Py_DEBUG *only* when you want to do the refcount debugging. You have to configure --with-pydebug for that, and you never do that in the wild. *If* you would install a pydebug binary, you would use a different --prefix, so that it would not interfere with the standard installation. Depending on which pyconfig.h extensions pick up, they would then be suitable for use with one or the other installation. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 02:37 Message: Logged In: YES user_id=14198 *sigh* - forgot that the object layout changed :( Re malloc issues: they are indeed still a problem - but Python will not *force* you to do this. It used to be the case that an extension module would malloc an object, and whatever module did the final Py_DECREF (often the core) would perform the free(). I believe this is no longer true. It *is* still true with FILE objects and the tp_print slot IIRC. My point with the "_d" was that iff we can get things working so that the standard Python extensions are "safe", "_d" need not imply anything other than "the preferred extension for debug builds of the core" - and even that is only for b/w compat and convenience reasons. How has Linux survived so long? I assume Py_DEBUG builds are simply never made in the "wild". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Sun Nov 11 21:23:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 13:23:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 13:23 Message: Logged In: YES user_id=21627 Why does the linker report no error if expat.dll is integrated into a debug build? Because the conflict is detected only for object files compiled with different settings; the linker doesn't look into DLLs to see what CRT they use. I believe MS documented all known problems with combining CRTs; this is an issue that occurs in many projects, and all aspects are well-understood. Since pyexpat doesn't export any CRT objects created by expat, there is no problem with using expat.dll in a debug build. Anonymous, given all this information, do you still think there is a bug in Python? If so, can you please explain from scratch what this problem is (stating only what you believe to be facts)? If not, I propose to close this report as "Won't fix". ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-11 11:49 Message: Logged In: YES user_id=31435 Martin, the MS docs also say the linker generates a warning if you mix CRT types -- but it doesn't for mixing debug Python with expat or Tcl/Tk. I don't always believe everything in the MS docs . But, Mark, I do believe them when they say you're on thin ice, and they really don't document the ways in which mixing CRTs can be disastrous (they give three specific examples, "just a few" of "many [unspecified] ways to get into trouble"). Mark, I like _d fine! It's better than MS's plain "d" convention (which I've seen other projects mimic), and so long as there *is* a reason that mixing debug and release builds can be catastrophic, it's Good to make it painfully obvious that something potentially catastrophic is going on (note that Anonymous starting getting in trouble when he "disabled the use of the pragma in config.h" -- we spent most of this bug report trying to convince him the pragma wasn't there just to irritate him ). M and M, I don't *know* that expat.dll and Tcl/Tk work fine with a debug Python. All I've ever done with expat.dll then is run our test suite, and that simply doesn't crash; and I probably haven't run any Tcl/Tk scripts under a debug Python in years. Mark, I suspect debug Python builds are extremely rare on Unix, even among Python developers. For example, for more than a year, I believe I'm the only one who has found test- suite failures specific to the debug build. But "compile for debugging" to Unixoids usually means some combination of not defining NDEBUG, retaining symbol info, and/or cranking down optimization. The Unix libraries don't care about that stuff -- there's nothing there as Draconian as the MS-specific _DEBUG effect on libraries. Martin, It Hurts to tell people not to use _DEBUG on Windows: as a prime example, the MS _DEBUG malloc+free are indispensable debugging tools, and indeed have caught bugs in Python that escaped even efence. Users want that for their code too. I don't know where this leaves us. Probably right where we started, except we know better that we're lost . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:18 Message: Logged In: YES user_id=21627 On Linux, it is common to include debug information into every library and executable; in particular, it is common to install python with debugging information. This is possible since debugging information can be combined with optimization almost seamless in gcc (distributors often strip the debugging information, though). So you use Py_DEBUG *only* when you want to do the refcount debugging. You have to configure --with-pydebug for that, and you never do that in the wild. *If* you would install a pydebug binary, you would use a different --prefix, so that it would not interfere with the standard installation. Depending on which pyconfig.h extensions pick up, they would then be suitable for use with one or the other installation. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 02:37 Message: Logged In: YES user_id=14198 *sigh* - forgot that the object layout changed :( Re malloc issues: they are indeed still a problem - but Python will not *force* you to do this. It used to be the case that an extension module would malloc an object, and whatever module did the final Py_DECREF (often the core) would perform the free(). I believe this is no longer true. It *is* still true with FILE objects and the tp_print slot IIRC. My point with the "_d" was that iff we can get things working so that the standard Python extensions are "safe", "_d" need not imply anything other than "the preferred extension for debug builds of the core" - and even that is only for b/w compat and convenience reasons. How has Linux survived so long? I assume Py_DEBUG builds are simply never made in the "wild". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Mon Nov 12 05:48:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 21:48:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-477557 ] email package needs examples Message-ID: Bugs item #477557, was opened at 2001-11-02 08:23 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477557&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: email package needs examples Initial Comment: Fill out the examples section of the email package documentation. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-11 21:48 Message: Logged In: YES user_id=12800 Examples have been added. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477557&group_id=5470 From noreply@sourceforge.net Mon Nov 12 06:33:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 11 Nov 2001 22:33:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Mon Nov 12 08:51:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 00:51:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-475951 ] HP-UX: Problem building socket Message-ID: Bugs item #475951, was opened at 2001-10-29 02:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: HP-UX: Problem building socket Initial Comment: On HP-UX 11, I get errors compiling Python 2.1.1 from source. It crashes when parsing sys/socket.h Here's the offending code: extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); When I switch from gcc to cc (native c compiler) the socket library compiles, but many other libs don't. (sidenote: when using cc, cc is noteably faster than gcc) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 00:51 Message: Logged In: NO 'sbsize_t' is an HPUX11i issue - possible solution see below. I can't test it, I have only 11 without 'i'. More important is the fact, that Python uses UNIX style sockets as far as i understand. HPUX offers two types of sockets: BSD-style via libc and UNIX style via libxnet. The HPUX11 python build is linked against libxti (not xnet) via libnsl. Compilation of socketmodule.c HAS TO BE DONE with _XOPEN_SOURCE_EXTENDED defined - see below. With this one gets rid of a lot of compiler warnings with socketmodule.c However, my poor C skills don't tell me if -lxnet should be added additionally at link time or not. Carl cmkleffner.gmx.de -------------- The following article on devresource is a must for all HPUX porters: http://devresource.hp.com/STKL/inhibitors.html 'Linux porting inhibitors' > socket flavors > > HP-UX supports two types of sockets - BSD 4.2 style sockets in libc and UNIX95/98 > style sockets in libxnet. Linux's socket is a UNIX98 style socket, so the path of least > resistance would be to use the UNIX95/98 style socket in HP-UX. However, since > socklen_t in Linux is typedef to int while socklen_t in HP-UX is typedef to size_t, some > additional reconciliation may still be needed. Note, socklen_t is used in places where > "len" objects are defined. An example of its use is shown below. > > To use UNIX95/98 style sockets in HP-UX, the libxnet library needs to be added to the > link line. Also, the macro, _XOPEN_SOURCE, needs to be defined, and the macro, > _XOPEN_SOURCE_EXTENDED, needs to be defined as 1. Refer to HP-UX manpage, > xopen_networking(7) for further details. http://gcc.gnu.org/ml/gcc-prs/2001-07/msg00527.html . . . . . > HPUX 11i is quite different from HPUX 11 because many of the > header files have been changed and new types are used such as > sbsize_t and bsize_t. http://gcc.gnu.org/ml/gcc-bugs/2001-09/msg00139.html > To: hpux at connect dot org dot uk, gcc-bugs at gcc dot gnu dot org > Subject: GCC 3.0.1 on HP-UX 11i > From: Remi COLLET > Date: Wed, 05 Sep 2001 17:40:33 +0200 > Organization: IUT de Reims - Departement Informatique > . . . . . > 3) non ANSI declaration of sendfile and sendpath (C++) in socket.h > > Copy socket.h in > /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/3.0.1/include/sys > > Replace (line 470) > > > inline sbsize_t sendfile(a,b,c,d,e,f) int a,b,f; off_t c; bsize_t d; const struct iovec * e; { return __sendfile64(a,b,c,d,e,f); } > > inline sbsize_t sendpath(a,b,c,d,e,f) int a,f; char *b; off_t c; bsize_t d; const struct iovec * e; { return __sendpath64(a,b,c,d,e,f); } > > > > > With: > > > inline sbsize_t sendfile(int a,int b,off_t c,bsize_t d,const struct iovec *e,int f) > > { return __sendfile64(a,b,c,d,e,f); } > > inline sbsize_t sendpath(int a,char *b,off_t c,bsize_t d,const struct iovec *e,int f) > > { return __sendpath64(a,b,c,d,e,f); } > > > > > Then, all works fine ! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-06 11:12 Message: Logged In: YES user_id=21627 The likely cause is that sbsize_t is not defined. Please try to find out what header file of HP-UX defines sbsize_t, and under what conditions? Outright including that header file probably is the wrong solution, so please also find out where it is included from, tracking it eventually back to socket.h. It appears that a certain #define is missing to cause a proper definition of sbsize_t. We must find out what that define is, and why it is not set when compiling with gcc. Also, could you please identify yourself? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-06 04:42 Message: Logged In: NO gcc --version is now 3.0.1 The problem however is still there: building '_socket' extension gcc -g -O2 -Wall -Wstrict-prototypes -fpic -I. -I/utmnt/ut/si/dv0216/Python-2.1.1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.1/socketmodule.o In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:145: /usr/include/sys/socket.h:439: parse error before "sendfile" /usr/include/sys/socket.h:439: parse error before "bsize_t" /usr/include/sys/socket.h:441: parse error before "sendpath" /usr/include/sys/socket.h:441: parse error before "bsize_t" /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_accept': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:795: warning: passing arg 3 of `accept' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getsockopt': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:978: warning: passing arg 5 of `getsockopt' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:992: warning: passing arg 5 of `getsockopt' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getsockname': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1188: warning: passing arg 3 of `getsockname' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_getpeername': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1218: warning: passing arg 3 of `getpeername' from incompatible pointer type /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c: In function `PySocketSock_recvfrom': /utmnt/ut/si/dv0216/Python-2.1.1/Modules/socketmodule.c:1376: warning: passing arg 6 of `recvfrom' from incompatible pointer type WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 Here are the offending lines from socket.h extern sbsize_t sendfile __((int, int, off_t, bsize_t, const struct iovec *, int)); extern sbsize_t sendpath __((int, char *, off_t, bsize_t, const struct iovec *, int)); However, my medium c skills tell me that the lines are ok. I cannot offer an explanation why gcc fails here. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-01 08:57 Message: Logged In: YES user_id=21627 gcc 2.95 does not support HP-UX 11; this appears to be an instance of that problem. The __va__list problem is not at all Python-specific; it occurs in many packages (just search Google for "HP-UX __va__list"). With such problems, there is no point into looking further into the problem; just drop the compiler. If you want to port Python to HP-UX and gcc 2.95, you are on your own. Notice that gcc 3.0.1 *does* support HP-UX 11; I recommend to upgrade. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-01 07:41 Message: Logged In: NO gcc version is 2.95.3 HP-UX hp18 B.11.11 U 9000/785 2011589119 both Python 2.1.1 and 2.2b1 do not compile ootb here. Error messages during compile are building '_socket' extension gcc -g -O3 -Wall -Wstrict-prototypes -fPIC -I. -I/utmnt/ut/si/dv0216/Python-2.2b1/./Include -I/usr/local/include -IInclude/ -c /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c -o build/temp.hp-ux-B.11.11-9000/785-2.2/socketmodule.o In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:42, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: `__va__list' redefined /usr/include/sys/stdsyms.h:422: warning: this is the location of the previous definition In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Include/Python.h:47, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:77: /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/string.h:26: warning: `__va__list' redefined /opt/gcc/lib/gcc-lib/hppa2.0n-hp-hpux11.00/2.95.3/include/stdio.h:30: warning: this is the location of the previous definition In file included from /usr/include/netdb.h:72, from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:152: /usr/include/sys/socket.h:439: parse error before `sendfile' /usr/include/sys/socket.h:439: parse error before `bsize_t' /usr/include/sys/socket.h:440: warning: data definition has no type or storage class /usr/include/sys/socket.h:441: parse error before `sendpath' /usr/include/sys/socket.h:441: parse error before `bsize_t' /usr/include/sys/socket.h:442: warning: data definition has no type or storage class /usr/include/sys/socket.h:456: parse error before `__sendfile64' /usr/include/sys/socket.h:456: parse error before `bsize_t' /usr/include/sys/socket.h:456: warning: data definition has no type or storage class /usr/include/sys/socket.h:457: parse error before `__sendpath64' /usr/include/sys/socket.h:457: parse error before `bsize_t' /usr/include/sys/socket.h:457: warning: data definition has no type or storage class /usr/include/sys/socket.h:459: parse error before `sendfile' /usr/include/sys/socket.h:459: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendfile': /usr/include/sys/socket.h:459: parse error before `bsize_t' /usr/include/sys/socket.h: At top level: /usr/include/sys/socket.h:460: parse error before `sendpath' /usr/include/sys/socket.h:460: warning: function declaration isn't a prototype /usr/include/sys/socket.h: In function `sendpath': /usr/include/sys/socket.h:460: parse error before `bsize_t' In file included from /utmnt/ut/si/dv0216/Python-2.2b1/Modules/socketmodule.c:238: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: At top level: /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:203: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:212: warning: function declaration isn't a prototype /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `freeaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:219: warning: implicit declaration of function `free' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `str_isnumber': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:231: warning: subscript has type `char' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c: In function `getaddrinfo': /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:345: warning: implicit declaration of function `atoi' /utmnt/ut/si/dv0216/Python-2.2b1/Modules/getaddrinfo.c:396: warning: implicit declaration of function `malloc' WARNING: building of extension "_socket" failed: command 'gcc' failed with exit status 1 I now thought that since the errors happen in a system header that compiles with cc and not gcc, why not copy to "socket.h" and delete the two offending lines. However, as you see, is still included (why ?). Another option is using cc just for this library, but I'm not sure, that the object files are compatible. Using cc alone creates a lot of non-working modules, although it is in (extended) ANSI mode. I also tried the HP Porting centre (http://hpux.asknet.de) but the Python 2.1 package there -just like mine- does crash on importing socket (too with 2.2b1) hp18: Python-2.2b1 % ./python Python 2.2b1 (#4, Nov 1 2001, 16:36:54) [C] on hp-uxB Type "help", "copyright", "credits" or "license" for more information. >>> import socket Traceback (most recent call last): File "", line 1, in ? File "/utmnt/ut/si/dv0216/Python-2.2b1/Lib/socket.py", line 41, in ? from _socket import * ImportError: No module named _socket >>> ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-10-30 00:56 Message: Logged In: YES user_id=21627 It would be helpful if you would provide all details in a report also. When you say "it crashes", did you really mean "it crashes" (i.e. with a core dump, machine reboot, or the like)? If the compiler merely rejected the code, it would be good to see what the error message was. Please also report version numbers: HP-UX version, gcc version, etc. Are you sure you are using a gcc built for your OS version? If the gcc doesn't get prototypes right, this often comes from having fixincluded header files of a different OS version. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 04:59 Message: Logged In: YES user_id=6380 Please try again with Python 2.2b1 (http://python.org/2.2/). We've done a lot of work and we've got at least one report that it now works out of the box on HP-UX 11. I'd like to hear if that solves your problems (and if 2.2b1 builds with either compiler). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475951&group_id=5470 From noreply@sourceforge.net Mon Nov 12 09:29:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 01:29:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-479981 ] socket failed then compiled with treads Message-ID: Bugs item #479981, was opened at 2001-11-09 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 Category: Threads Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket failed then compiled with treads Initial Comment: Operating System HP-UX 11.0 (64 Bit) I tried to compile Python 2.1.1 this thread support. Doing so, the tests test_socket and test_asynchat both failed with "(9, 'Bad file number')" The test_socket succeds if I compiled python without threads, so I think it's an thread-problem. The thread-test istself succeeds. I think also, that python is correctly linked against libpthread. The same Problem occurs this Python 2.2.b1. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 01:29 Message: Logged In: NO the complete compiler line used to link together the python binary: gcc -Wl,-E -Wl,+s -Wl,+b/mosquito/python/lib/python2.1/lib- dynload -o python \ Modules/python.o \ libpython2.1.a -lnsl -ldld -lpthread -lm The compiler I used is gcc 3.01. I tried it out with the cc from HP - the same effect After installing the Kernelpatch PHKL_25475, wich supersedes both PHKL_23842 and PHKL_20942 the failure remains the same ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 16:13 Message: Logged In: YES user_id=21627 Can you please also check whether the patches PHKL_23842 (or equivalent) and/or PHKL_20942. The patch titles are PHKL_23842 s700_800 11.00 signal,threads,spinlock,scheduler,IDS,q3p PHKL_20942 s700_800 11.00 signal, nanosleep, spinlock, scheduler It appears that HP-UX has a number of bugs where signals, forking, and other aspects of the operating functionality stop working in the presence of threads. If you don't have these patches, it would be good if you could try to install them, and report back whether it changes anything. Notice that HP apparently has a number of patches with these titles; they seem to be intended for different machines. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 15:51 Message: Logged In: YES user_id=21627 What compiler are you using? Can you please report the complete compiler line used to link together the python binary? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 From noreply@sourceforge.net Mon Nov 12 10:12:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 02:12:07 -0800 Subject: [Python-bugs-list] [ python-Bugs-479981 ] socket failed then compiled with treads Message-ID: Bugs item #479981, was opened at 2001-11-09 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 Category: Threads Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket failed then compiled with treads Initial Comment: Operating System HP-UX 11.0 (64 Bit) I tried to compile Python 2.1.1 this thread support. Doing so, the tests test_socket and test_asynchat both failed with "(9, 'Bad file number')" The test_socket succeds if I compiled python without threads, so I think it's an thread-problem. The thread-test istself succeeds. I think also, that python is correctly linked against libpthread. The same Problem occurs this Python 2.2.b1. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 02:12 Message: Logged In: NO Maybe the failure has to do with gethostbyname-stuff. according the hpux-manpages theese reentrant interfaces are obsolete: int gethostent_r(struct hostent *result, struct hostent_data *buffer); int gethostbyname_r(const char *name, struct hostent *result, struct hostent_data *buffer); int gethostbyaddr_r(const char *addr, int len, int type, struct hostent *result, struct hostent_data *buffer); int sethostent_r(int stayopen, struct hostent_data *buffer); int endhostent_r(struct hostent_data *buffer); The configure-script has correctly found, that there is only a gethostbyname, but no gethostbyname_r. according to the man-pages the call to gethostbyname IS thread-save: struct hostent *gethostent(void); struct hostent *gethostbyname(const char *name); struct hostent *gethostbyaddr(const char *addr, int len, int type); ... MULTITHREAD USAGE Thread Safe: Yes Cancel Safe: Yes Async-cancel Safe: No Async-signal Safe: No Is it a failure to expect the gethostbyname to be not thread-save ? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 01:29 Message: Logged In: NO the complete compiler line used to link together the python binary: gcc -Wl,-E -Wl,+s -Wl,+b/mosquito/python/lib/python2.1/lib- dynload -o python \ Modules/python.o \ libpython2.1.a -lnsl -ldld -lpthread -lm The compiler I used is gcc 3.01. I tried it out with the cc from HP - the same effect After installing the Kernelpatch PHKL_25475, wich supersedes both PHKL_23842 and PHKL_20942 the failure remains the same ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 16:13 Message: Logged In: YES user_id=21627 Can you please also check whether the patches PHKL_23842 (or equivalent) and/or PHKL_20942. The patch titles are PHKL_23842 s700_800 11.00 signal,threads,spinlock,scheduler,IDS,q3p PHKL_20942 s700_800 11.00 signal, nanosleep, spinlock, scheduler It appears that HP-UX has a number of bugs where signals, forking, and other aspects of the operating functionality stop working in the presence of threads. If you don't have these patches, it would be good if you could try to install them, and report back whether it changes anything. Notice that HP apparently has a number of patches with these titles; they seem to be intended for different machines. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 15:51 Message: Logged In: YES user_id=21627 What compiler are you using? Can you please report the complete compiler line used to link together the python binary? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 From noreply@sourceforge.net Mon Nov 12 12:02:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 04:02:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-478421 ] cPickle bug when using imp.load_source() Message-ID: Bugs item #478421, was opened at 2001-11-05 12:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478421&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Andrew R Gross (sincere2001) Assigned to: Nobody/Anonymous (nobody) Summary: cPickle bug when using imp.load_source() Initial Comment: When a Python 2.1.1 program dynamically creates a module using the 'imp.load_source()' function, and the created module tries to 'import cPickle', the interpreter dies with a SystemError exception: SystemError: _PyImport_FixupExtension: module import_cPickle.cPickle not loaded This does *not* occur when the find/load_module() functions are used. The bug also appears to have been introduced somewhere in 2.x, as 1.5.x works fine with load_source(). Attached is a full explanation of the behavior and some small sample programs to illustrate the bug. ---------------------------------------------------------------------- Comment By: Simo Salminen (frangen) Date: 2001-11-12 04:02 Message: Logged In: YES user_id=291461 Your example works fine with my system. However: You are giving imp.load_source bad 'name' paramater. In your example: mod2 = imp.load_source(file, file, open(file)) Change that first paramater to os.path.splitext(file)[0] and it might work. Here is an example program that shows how load_source fails with bad first paramater: egg.py: import imp imp.load_source('spam.py', 'spam.py', open('spam.py')) spam.py: import os.path % python egg.py ImportError: No module named path and now fix that egg.py file: imp.load_source('spam', 'spam.py', open('spam.py')) -> works fine. (tested with activepython 2.1.1) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478421&group_id=5470 From noreply@sourceforge.net Mon Nov 12 12:19:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 04:19:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-480882 ] oot builds don't build _curses_panel Message-ID: Bugs item #480882, was opened at 2001-11-12 04:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480882&group_id=5470 Category: Build Group: None Status: Open Resolution: None Priority: 5 Submitted By: Michael Hudson (mwh) Assigned to: A.M. Kuchling (akuchling) Summary: oot builds don't build _curses_panel Initial Comment: Out-of-tree builds of python (i.e. $ cd PATH/TO/Python/src $ mkdir build $ cd ./build $ ../configure && make ) don't build the _curses_panel extension. I think this line from setup.py: if (os.path.exists('Modules/_curses_panel.c') and ... is why. Either (1) this line should be fixed to actually point at the source (2) this check should simply be hacked out. What's it doing there? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480882&group_id=5470 From noreply@sourceforge.net Mon Nov 12 14:18:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 06:18:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 06:18 Message: Logged In: NO (Not anonymous: Jeff Kotula, Vital Images, Inc.) "Bug" is probably not the right classification. I believe it is a weakness in the configuration management though. Here's the problem: As a client of the C API, I want to be able to create a static library that is compiled with DEBUG and/or _DEBUG and be able to select either the debug or non-debug version of the python library independently. This helps me stay as close as possible to production-level configuration even when debugging some parts of the system. >From your preceding discussion it sounds like the only real barrier to this is the coupling of _DEBUG to Py_DEBUG. If this is the case, it should be a small matter to decouple the two. Changing the layout of objects could also be avoided simply by always including the reference counting field -- would it be that big of a percentage increase in object size? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 13:23 Message: Logged In: YES user_id=21627 Why does the linker report no error if expat.dll is integrated into a debug build? Because the conflict is detected only for object files compiled with different settings; the linker doesn't look into DLLs to see what CRT they use. I believe MS documented all known problems with combining CRTs; this is an issue that occurs in many projects, and all aspects are well-understood. Since pyexpat doesn't export any CRT objects created by expat, there is no problem with using expat.dll in a debug build. Anonymous, given all this information, do you still think there is a bug in Python? If so, can you please explain from scratch what this problem is (stating only what you believe to be facts)? If not, I propose to close this report as "Won't fix". ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-11 11:49 Message: Logged In: YES user_id=31435 Martin, the MS docs also say the linker generates a warning if you mix CRT types -- but it doesn't for mixing debug Python with expat or Tcl/Tk. I don't always believe everything in the MS docs . But, Mark, I do believe them when they say you're on thin ice, and they really don't document the ways in which mixing CRTs can be disastrous (they give three specific examples, "just a few" of "many [unspecified] ways to get into trouble"). Mark, I like _d fine! It's better than MS's plain "d" convention (which I've seen other projects mimic), and so long as there *is* a reason that mixing debug and release builds can be catastrophic, it's Good to make it painfully obvious that something potentially catastrophic is going on (note that Anonymous starting getting in trouble when he "disabled the use of the pragma in config.h" -- we spent most of this bug report trying to convince him the pragma wasn't there just to irritate him ). M and M, I don't *know* that expat.dll and Tcl/Tk work fine with a debug Python. All I've ever done with expat.dll then is run our test suite, and that simply doesn't crash; and I probably haven't run any Tcl/Tk scripts under a debug Python in years. Mark, I suspect debug Python builds are extremely rare on Unix, even among Python developers. For example, for more than a year, I believe I'm the only one who has found test- suite failures specific to the debug build. But "compile for debugging" to Unixoids usually means some combination of not defining NDEBUG, retaining symbol info, and/or cranking down optimization. The Unix libraries don't care about that stuff -- there's nothing there as Draconian as the MS-specific _DEBUG effect on libraries. Martin, It Hurts to tell people not to use _DEBUG on Windows: as a prime example, the MS _DEBUG malloc+free are indispensable debugging tools, and indeed have caught bugs in Python that escaped even efence. Users want that for their code too. I don't know where this leaves us. Probably right where we started, except we know better that we're lost . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:18 Message: Logged In: YES user_id=21627 On Linux, it is common to include debug information into every library and executable; in particular, it is common to install python with debugging information. This is possible since debugging information can be combined with optimization almost seamless in gcc (distributors often strip the debugging information, though). So you use Py_DEBUG *only* when you want to do the refcount debugging. You have to configure --with-pydebug for that, and you never do that in the wild. *If* you would install a pydebug binary, you would use a different --prefix, so that it would not interfere with the standard installation. Depending on which pyconfig.h extensions pick up, they would then be suitable for use with one or the other installation. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 02:37 Message: Logged In: YES user_id=14198 *sigh* - forgot that the object layout changed :( Re malloc issues: they are indeed still a problem - but Python will not *force* you to do this. It used to be the case that an extension module would malloc an object, and whatever module did the final Py_DECREF (often the core) would perform the free(). I believe this is no longer true. It *is* still true with FILE objects and the tp_print slot IIRC. My point with the "_d" was that iff we can get things working so that the standard Python extensions are "safe", "_d" need not imply anything other than "the preferred extension for debug builds of the core" - and even that is only for b/w compat and convenience reasons. How has Linux survived so long? I assume Py_DEBUG builds are simply never made in the "wild". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Mon Nov 12 17:27:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 09:27:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-480325 ] Tkinter crash on Win2000, OK on UNIX Message-ID: Bugs item #480325, was opened at 2001-11-10 01:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 Category: Tkinter Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: John W Lewis (lewisjwl) Assigned to: Nobody/Anonymous (nobody) Summary: Tkinter crash on Win2000, OK on UNIX Initial Comment: Large Python/Tkinter application.... 2GB virtual, 3600 images, 3600 graphic objects, 3600x3600 arrays 3200x1600 dual monitor graphics Runs OK under 2.1.1 Python and Solaris8 Crashes in TK on Windows2000 at 3200 images DELL 530 workstation with 4GB RAM Windows 2000 SR1 2.1.1 Python How do I go about isolating this problem? Thanks John Lewis ---------------------------------------------------------------------- >Comment By: John W Lewis (lewisjwl) Date: 2001-11-12 09:27 Message: Logged In: YES user_id=89001 >From the dump file.... Microsoft (R) WIndows 2000 (TM) Version 5.00 DrWtsn32 Application exception occured App: (pid=1052) Exception number: c0000005 (access violation) Number of processors: 2 Processor Type x86 family 15 Model 0 Stepping 10 Windows 2000 VErsion: 5.0 Current Build 2195 Service Pack: 1 Current Type: Multiprocessor Free !TkSetPixmapColormap !Tk_CreatePhotoImageFormat !Tk_CreatePhotoImageFormat !Tk_CreatePhotoImageFormat !Tk_GetImage !Tk_InvokeButton !Tk_InvokeButton !Tk_CanvasObjCmd !Tcl_EvalObjv !Tcl_EvalObjv ! Is this helpful? John ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 06:07 Message: Logged In: YES user_id=21627 I recommend to attach to the process with a debugger (e.g. VC++). If you manage to do this, please report the stack trace at the moment of the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 From noreply@sourceforge.net Mon Nov 12 18:14:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 10:14:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Mon Nov 12 18:52:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 10:52:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Tue Nov 13 00:38:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 16:38:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Tue Nov 13 04:52:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 20:52:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-12 20:52 Message: Logged In: YES user_id=31435 Martin, where does MS document "all known problems with combining CRTs"? The KB article you referenced before (Q140584) says "There are many ways to get into trouble with two CRTs. Here are just a few:", and then lists the same three specific examples that have been brought up in this bug report. There's not a clue there about what the rest of the many ways may be. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 06:18 Message: Logged In: NO (Not anonymous: Jeff Kotula, Vital Images, Inc.) "Bug" is probably not the right classification. I believe it is a weakness in the configuration management though. Here's the problem: As a client of the C API, I want to be able to create a static library that is compiled with DEBUG and/or _DEBUG and be able to select either the debug or non-debug version of the python library independently. This helps me stay as close as possible to production-level configuration even when debugging some parts of the system. >From your preceding discussion it sounds like the only real barrier to this is the coupling of _DEBUG to Py_DEBUG. If this is the case, it should be a small matter to decouple the two. Changing the layout of objects could also be avoided simply by always including the reference counting field -- would it be that big of a percentage increase in object size? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 13:23 Message: Logged In: YES user_id=21627 Why does the linker report no error if expat.dll is integrated into a debug build? Because the conflict is detected only for object files compiled with different settings; the linker doesn't look into DLLs to see what CRT they use. I believe MS documented all known problems with combining CRTs; this is an issue that occurs in many projects, and all aspects are well-understood. Since pyexpat doesn't export any CRT objects created by expat, there is no problem with using expat.dll in a debug build. Anonymous, given all this information, do you still think there is a bug in Python? If so, can you please explain from scratch what this problem is (stating only what you believe to be facts)? If not, I propose to close this report as "Won't fix". ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-11 11:49 Message: Logged In: YES user_id=31435 Martin, the MS docs also say the linker generates a warning if you mix CRT types -- but it doesn't for mixing debug Python with expat or Tcl/Tk. I don't always believe everything in the MS docs . But, Mark, I do believe them when they say you're on thin ice, and they really don't document the ways in which mixing CRTs can be disastrous (they give three specific examples, "just a few" of "many [unspecified] ways to get into trouble"). Mark, I like _d fine! It's better than MS's plain "d" convention (which I've seen other projects mimic), and so long as there *is* a reason that mixing debug and release builds can be catastrophic, it's Good to make it painfully obvious that something potentially catastrophic is going on (note that Anonymous starting getting in trouble when he "disabled the use of the pragma in config.h" -- we spent most of this bug report trying to convince him the pragma wasn't there just to irritate him ). M and M, I don't *know* that expat.dll and Tcl/Tk work fine with a debug Python. All I've ever done with expat.dll then is run our test suite, and that simply doesn't crash; and I probably haven't run any Tcl/Tk scripts under a debug Python in years. Mark, I suspect debug Python builds are extremely rare on Unix, even among Python developers. For example, for more than a year, I believe I'm the only one who has found test- suite failures specific to the debug build. But "compile for debugging" to Unixoids usually means some combination of not defining NDEBUG, retaining symbol info, and/or cranking down optimization. The Unix libraries don't care about that stuff -- there's nothing there as Draconian as the MS-specific _DEBUG effect on libraries. Martin, It Hurts to tell people not to use _DEBUG on Windows: as a prime example, the MS _DEBUG malloc+free are indispensable debugging tools, and indeed have caught bugs in Python that escaped even efence. Users want that for their code too. I don't know where this leaves us. Probably right where we started, except we know better that we're lost . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:18 Message: Logged In: YES user_id=21627 On Linux, it is common to include debug information into every library and executable; in particular, it is common to install python with debugging information. This is possible since debugging information can be combined with optimization almost seamless in gcc (distributors often strip the debugging information, though). So you use Py_DEBUG *only* when you want to do the refcount debugging. You have to configure --with-pydebug for that, and you never do that in the wild. *If* you would install a pydebug binary, you would use a different --prefix, so that it would not interfere with the standard installation. Depending on which pyconfig.h extensions pick up, they would then be suitable for use with one or the other installation. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 02:37 Message: Logged In: YES user_id=14198 *sigh* - forgot that the object layout changed :( Re malloc issues: they are indeed still a problem - but Python will not *force* you to do this. It used to be the case that an extension module would malloc an object, and whatever module did the final Py_DECREF (often the core) would perform the free(). I believe this is no longer true. It *is* still true with FILE objects and the tp_print slot IIRC. My point with the "_d" was that iff we can get things working so that the standard Python extensions are "safe", "_d" need not imply anything other than "the preferred extension for debug builds of the core" - and even that is only for b/w compat and convenience reasons. How has Linux survived so long? I assume Py_DEBUG builds are simply never made in the "wild". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Tue Nov 13 05:43:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 21:43:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-481171 ] Hang using files named prn.txt, etc Message-ID: Bugs item #481171, was opened at 2001-11-12 21:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481171&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Andrew Bennetts (spiv) Assigned to: Tim Peters (tim_one) Summary: Hang using files named prn.txt, etc Initial Comment: Windows reserves many possible filenames as reserved device names. Examples are prn.xxx, con.xxx, com1.xxx aux.xxx, etc, where xxx can be any extension. Attempts to create/read/write to these files can hang the interpreter, but ideally would just throw an OSError. A partial list of device names is mentioned at: http://cert.uni- stuttgart.de/archive/bugtraq/2001/07/msg00086.html ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481171&group_id=5470 From noreply@sourceforge.net Tue Nov 13 06:18:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 12 Nov 2001 22:18:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Tue Nov 13 08:17:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 00:17:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-481218 ] popen4 fails if cmd is unicode. Message-ID: Bugs item #481218, was opened at 2001-11-13 00:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481218&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: popen4 fails if cmd is unicode. Initial Comment: In popen2.py there is a class Popen3. It contains a method _run_child(). ------- def _run_child(self, cmd): if type(cmd) == type(''): cmd = ['/bin/sh', '-c', cmd] ------- I think that this test is attempting to find if the "cmd" is not a list. Unfortunately does not get turned into a list. A better test may be: ----- def _run_child(self, cmd): if repr(type(cmd)) != "" cmd = ['/bin/sh', '-c', cmd] ----- This problem shows up when creating commands from XML files read by "xml.sax". Beta 2.2b1 still has this code. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481218&group_id=5470 From noreply@sourceforge.net Tue Nov 13 08:54:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 00:54:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-481221 ] rfc822.Addrlist class fails on long addr Message-ID: Bugs item #481221, was opened at 2001-11-13 00:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ondrej Palkovsky (ondrap) Assigned to: Nobody/Anonymous (nobody) Summary: rfc822.Addrlist class fails on long addr Initial Comment: The Addrlistclass.getaddrlist uses recursion algorithm, unfortunately on some _very_ large address fields it exceeds the maximum recursion. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 From noreply@sourceforge.net Tue Nov 13 08:56:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 00:56:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-480325 ] Tkinter crash on Win2000, OK on UNIX Message-ID: Bugs item #480325, was opened at 2001-11-10 01:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 Category: Tkinter Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: John W Lewis (lewisjwl) Assigned to: Nobody/Anonymous (nobody) Summary: Tkinter crash on Win2000, OK on UNIX Initial Comment: Large Python/Tkinter application.... 2GB virtual, 3600 images, 3600 graphic objects, 3600x3600 arrays 3200x1600 dual monitor graphics Runs OK under 2.1.1 Python and Solaris8 Crashes in TK on Windows2000 at 3200 images DELL 530 workstation with 4GB RAM Windows 2000 SR1 2.1.1 Python How do I go about isolating this problem? Thanks John Lewis ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-13 00:56 Message: Logged In: YES user_id=21627 It helps to a degree, but only insofar as recognizing this as a Tk problem. If you have the Tk source, you may want to follow what I present as a theory: In generic/tkImgPhoto.c:ImgPhotoInstanceSetSize, it invokes Tk_GetPixmap. This is implemented in win/tkWinPixmap.c. If CreateBitmap fails, it will return None, which is 0. So back in ImgPhotoInstanceSetSize, it will pass the null pointer to TkSetPixmapColormap, which is again implemented in tkWinPixmap.c, where it says twdPtr->bitmap.colormap = colormap; Since twdPtr comes from pixmap, which comes from newPixmap, which may be null, this is a null-pointer access, which causes the Access Violation. Since this is the only action in TkSetPixmapColormap, and since ImgPhotoInstanceSetSize is the only place where TkSetPixmapColormap, this theory is quite likely, IMO, but only single-stepping in a debugger may tell for sure. In any case, it seems that CreateBitmap fails, which means that you are running out of bitmap handles. To my knowledge, on Win32, this only happens when you run out of memory; a Windows expert may be able to give a more detailed analysis. I see two problems: - why doesn't Tk report the error properly? It doesn't always check the return value, which it should; please report that as a Tk bug: This is nothing we can do much about; we won't go into the business of fixing Tk for ourselves. - why does it run out of bitmap handles? This depends on whether it is supposed to run out of bitmap handles: If it is out of memory, there is not much you could do - it is an application bug to allocate so many of them. If you are not out of memory, why does it fail to create more bitmaps? That would be a Windows bug; please report it to Microsoft :-) Perhaps MS has documented limitations on creating too many bitmaps; if so, the ball is back on Tk's side, which should use bitmap handles more conservatively, or on the application's side, which should not create so many bitmaps. Perhaps this is something completely different; only a debugger run could tell. ---------------------------------------------------------------------- Comment By: John W Lewis (lewisjwl) Date: 2001-11-12 09:27 Message: Logged In: YES user_id=89001 >From the dump file.... Microsoft (R) WIndows 2000 (TM) Version 5.00 DrWtsn32 Application exception occured App: (pid=1052) Exception number: c0000005 (access violation) Number of processors: 2 Processor Type x86 family 15 Model 0 Stepping 10 Windows 2000 VErsion: 5.0 Current Build 2195 Service Pack: 1 Current Type: Multiprocessor Free !TkSetPixmapColormap !Tk_CreatePhotoImageFormat !Tk_CreatePhotoImageFormat !Tk_CreatePhotoImageFormat !Tk_GetImage !Tk_InvokeButton !Tk_InvokeButton !Tk_CanvasObjCmd !Tcl_EvalObjv !Tcl_EvalObjv ! Is this helpful? John ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 06:07 Message: Logged In: YES user_id=21627 I recommend to attach to the process with a debugger (e.g. VC++). If you manage to do this, please report the stack trace at the moment of the crash. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480325&group_id=5470 From noreply@sourceforge.net Tue Nov 13 09:09:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 01:09:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-478339 ] Linking/compiling with DEBUG Message-ID: Bugs item #478339, was opened at 2001-11-05 07:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 Category: Windows Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: Linking/compiling with DEBUG Initial Comment: I have an embedded use of the Python interpreter. I want to compile and link the encapsulating application with and without DEBUG. I have disabled the use of the pragma in config.h to always use the library specified in the link statement, which is always the standard python library. I have ensured that all invocations of Python.h undefine the DEBUG symbol before including them. Compiling the application without debug: works fine if you link non-debug. However, linking with debug causes some change in the way python modules are looked up and I'm getting undefined local modules. I have checked the PYTHONPATH environment variable and it is correct, and the .py module exists in the correct directory. (This is a common scenario where some libraries are compiled without DEBUG, some with DEBUG, and then the whole application linked DEBUG.) Compiling the application with debug: Link errors result. Undefined -- _Py_RefTotal, _Py_Dealloc, _PyInitModule4TraceRefs. Python is making too many hidden assumptions about the build environment when it is being used embedded. This is a critical problem as it makes it impossible to build the encapsulating application for debug purposes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-13 01:09 Message: Logged In: YES user_id=21627 MS doesn't document it as a complete list; I still believe it is a complete list. Of course, you have no reason to trust me more than you trust Microsoft :-) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 20:52 Message: Logged In: YES user_id=31435 Martin, where does MS document "all known problems with combining CRTs"? The KB article you referenced before (Q140584) says "There are many ways to get into trouble with two CRTs. Here are just a few:", and then lists the same three specific examples that have been brought up in this bug report. There's not a clue there about what the rest of the many ways may be. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 06:18 Message: Logged In: NO (Not anonymous: Jeff Kotula, Vital Images, Inc.) "Bug" is probably not the right classification. I believe it is a weakness in the configuration management though. Here's the problem: As a client of the C API, I want to be able to create a static library that is compiled with DEBUG and/or _DEBUG and be able to select either the debug or non-debug version of the python library independently. This helps me stay as close as possible to production-level configuration even when debugging some parts of the system. >From your preceding discussion it sounds like the only real barrier to this is the coupling of _DEBUG to Py_DEBUG. If this is the case, it should be a small matter to decouple the two. Changing the layout of objects could also be avoided simply by always including the reference counting field -- would it be that big of a percentage increase in object size? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 13:23 Message: Logged In: YES user_id=21627 Why does the linker report no error if expat.dll is integrated into a debug build? Because the conflict is detected only for object files compiled with different settings; the linker doesn't look into DLLs to see what CRT they use. I believe MS documented all known problems with combining CRTs; this is an issue that occurs in many projects, and all aspects are well-understood. Since pyexpat doesn't export any CRT objects created by expat, there is no problem with using expat.dll in a debug build. Anonymous, given all this information, do you still think there is a bug in Python? If so, can you please explain from scratch what this problem is (stating only what you believe to be facts)? If not, I propose to close this report as "Won't fix". ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-11 11:49 Message: Logged In: YES user_id=31435 Martin, the MS docs also say the linker generates a warning if you mix CRT types -- but it doesn't for mixing debug Python with expat or Tcl/Tk. I don't always believe everything in the MS docs . But, Mark, I do believe them when they say you're on thin ice, and they really don't document the ways in which mixing CRTs can be disastrous (they give three specific examples, "just a few" of "many [unspecified] ways to get into trouble"). Mark, I like _d fine! It's better than MS's plain "d" convention (which I've seen other projects mimic), and so long as there *is* a reason that mixing debug and release builds can be catastrophic, it's Good to make it painfully obvious that something potentially catastrophic is going on (note that Anonymous starting getting in trouble when he "disabled the use of the pragma in config.h" -- we spent most of this bug report trying to convince him the pragma wasn't there just to irritate him ). M and M, I don't *know* that expat.dll and Tcl/Tk work fine with a debug Python. All I've ever done with expat.dll then is run our test suite, and that simply doesn't crash; and I probably haven't run any Tcl/Tk scripts under a debug Python in years. Mark, I suspect debug Python builds are extremely rare on Unix, even among Python developers. For example, for more than a year, I believe I'm the only one who has found test- suite failures specific to the debug build. But "compile for debugging" to Unixoids usually means some combination of not defining NDEBUG, retaining symbol info, and/or cranking down optimization. The Unix libraries don't care about that stuff -- there's nothing there as Draconian as the MS-specific _DEBUG effect on libraries. Martin, It Hurts to tell people not to use _DEBUG on Windows: as a prime example, the MS _DEBUG malloc+free are indispensable debugging tools, and indeed have caught bugs in Python that escaped even efence. Users want that for their code too. I don't know where this leaves us. Probably right where we started, except we know better that we're lost . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 06:18 Message: Logged In: YES user_id=21627 On Linux, it is common to include debug information into every library and executable; in particular, it is common to install python with debugging information. This is possible since debugging information can be combined with optimization almost seamless in gcc (distributors often strip the debugging information, though). So you use Py_DEBUG *only* when you want to do the refcount debugging. You have to configure --with-pydebug for that, and you never do that in the wild. *If* you would install a pydebug binary, you would use a different --prefix, so that it would not interfere with the standard installation. Depending on which pyconfig.h extensions pick up, they would then be suitable for use with one or the other installation. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 02:37 Message: Logged In: YES user_id=14198 *sigh* - forgot that the object layout changed :( Re malloc issues: they are indeed still a problem - but Python will not *force* you to do this. It used to be the case that an extension module would malloc an object, and whatever module did the final Py_DECREF (often the core) would perform the free(). I believe this is no longer true. It *is* still true with FILE objects and the tp_print slot IIRC. My point with the "_d" was that iff we can get things working so that the standard Python extensions are "safe", "_d" need not imply anything other than "the preferred extension for debug builds of the core" - and even that is only for b/w compat and convenience reasons. How has Linux survived so long? I assume Py_DEBUG builds are simply never made in the "wild". ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 02:27 Message: Logged In: YES user_id=21627 I thought the problem with memory blocks still exists: If you malloc with the debug CRT, and free with the nodebug CRT, you get a leak. At least Q140584 claims the problem still exists in 6.0sp5. In addition to memory and files, they also notice that setting the locale in one CRT doesn't affect the other. I still cannot tell whether Anonymous builds a DLL wrapper or a static library around pythonxy.dll.If it was a DLL, it would be indeed equivalent to the pyexpat case. However, in that case, linking Python wouldn't matter at all to the clients of his or her DLL, since the reference to pythonxy.dll/pythonxy_d.dll would be only in the wrapper DLL. Re: decoupling _DEBUG and Py_DEBUG. How exactly would that work? Would a _d extension imply _DEBUG, or Py_DEBUG, or would we stop using _d? If it would imply _DEBUG, I cannot understand Mark's comment "Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable." You have to use Py_DEBUG consistently throughout all modules because it changes the struct _object layout. Mixing them certainly isn't stable. Perhaps we could try to completely avoid changing PyObject_HEAD depending on Py_TRACE_REFS, and make it a type flag instead. In that case, you could, at run-time, mix objects that do trace_refs, and objects that don't. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-11 01:15 Message: Logged In: YES user_id=14198 The problem only occurs when the 2 DLLs must cooperate WRT CRT resources. Eg, if a "FILE *" was ever passed and used across the expat->Python boundary, or a memory block allocated in one to be freed in the other, then we would be in the same situation. This probably doesn't happen for expat and Tk. For Python and its extension modules (and even embedders), it may. IIRC, the problem with memory blocks was solved some time ago, leaving FILE* the only real problem. File objects are used in only 1 or 2 cases, and these could be solved by moving to "PyFile_Write()" style APIs. So, I guess I could agree a good solution is to ensure that Python C API does not require CRT resource sharing, and that Py_DEBUG is defined independently of _DEBUG or DEBUG. Individual extensions would be responsible for setting Py_DEBUG via their makefiles in debug builds. This should be stable. This would blur the _d convention. I quite like it, but I agree it is a PITA for many people. We could try giving pythonxx_d.dll the ability to load non _d versions, in the hope that the extensions have been "fixed" so they don't directly share the CRT. Tim - never quite understood your feelings on _d - what are your thoughts? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-10 22:15 Message: Logged In: YES user_id=31435 Martin and Mark, I want to play devil's advocate here just a bit: How do we get away with, e.g., using expat.dll in Windows Python? That is, we use the same expat.dll with the release and debug Python builds. Nothing bad happens that I've seen, and we don't get warnings about mixing CRT types either. IOW, we use expat.dll the way Anonymous would like to use the Python DLL. How do we get away with that? If that one is easy , ee don't use different versions of the Tcl/Tk libraries either. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-10 02:13 Message: Logged In: YES user_id=21627 The second rule should read Linking with python_d -> compile client code with _DEBUG since Python does not care whether DEBUG is defined or not; it only cares about _DEBUG (i.e. /MDd, as you point out). Furthermore, the second dependcy is also thanks to Microsoft: If you compile your library with /MDd, you must also compile all client code with /MDd, see MSKB Q140584). Python currently supports two configurations on Windows: 1. Neither of _DEBUG, Py_DEBUG is defined; link with pythonxy.dll and msvcrt.dll (i.e. /MD); extensions end with .pyd or .dll 2. Both _DEBUG, Py_DEBUG are defined, link with pythonxy_d.dll and msvcrtd.dll, extensions end with _d.pyd or _d.dll. It would be possible to create a third configuration: _DEBUG is defined, but Py_DEBUG isn't. Link with a no-debug pythonxy.dll, but with msvcrtd.dll. You couldn't use the existing pythonxy.dll and existing extension modules, since those already link with msvcrt.dll. Adding such a configuration would increase the mess, so it won't happen. I'd advise you to pursue a fourth configuration: Do not use the debug CRT (i.e. compile with /MD instead of /MDd), even if you compile your own library for debugging. With that configuration, you can use the standard Python distribution with no changes. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 11:30 Message: Logged In: NO Turns out the _DEBUG is defined when using the /MDd compiler option. We have painstakingly worked out our build system to deal with the library conflicts you mention. If I understand what you're saying, these are the rules: Linking with debug CRT -> link with python_d Linking with python_d -> compile client code with DEBUG The first rule is thanks to MS. The second rule is to get internal object layouts to match up. Is this right? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 10:17 Message: Logged In: YES user_id=21627 I'm glad the PYTHONPATH issue is resolved. On the _DEBUG issue: Somebody *must* be defining _DEBUG somewhere, or else the #ifdef would not trigger. Please open the MSVC++ project file of your project (the .dsp file) in a text editor, and search for _DEBUG. If you cannot find it there, please try to activate full command line printing in your build process, so that you see what options are passed to the compiler. If you are absolutely sure that _DEBUG is not passed in the command line, it must be defined in one of the header files. In this case, please put the #ifdef _DEBUG block of the previous test after each #include, e.g. in a form #include #ifdef _DEBUG #error After stdio.h #endif #include #ifdef _DEBUG #error After nextfile.h #endif I'm sure there are better ways to find out where _DEBUG is defined; don't hesitate to apply them if you are aware of any. Once you found the location of _DEBUG, you may remove it to be able to use the standard Python .DLL. Please make sure you understand Mark's comment on CRTs, though: Many problems will occur if you mix the debug and the nodebug version of the MSVC libraries (e.g. msvcrt.dll and msvcrtd.dll). Please see the Microsoft documentation for details: If you use pythonxy.dll, you absolutely *must* make sure that you do not link the debug versions of the CRT, even when performing a debug build. Unfortunately, you won't get a compilation error if you do not follow this guideline; instead, very hard-to-debug problems will occur. You can use depends.exe on the final executable to analyse the set of libraries used. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 08:16 Message: Logged In: NO Compiling without debug worked fine with the additional #ifdefs. Compiling with DEBUG resulted in the _DEBUG message and the Py_Debug message. sys.path seems to be ok now in all configurations. One of the (many) experimental changes to the build I did must have bypassed the problem. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 07:51 Message: Logged In: YES user_id=21627 In your code that includes Python.h, can you please add the lines #ifdef _DEBUG #error You must not define _DEBUG #endif #ifdef Py_DEBUG #error For some reason Py_DEBUG is still defined #endif immediately after the #include line? Then please compile your library with DEBUG, and report whether it compiles correctly; if not, please report the errors it gives. Furthermore, could you please evaluate the string "import sys;print sys.path" in your embedded code and report the exact output that this gives when running with and without DEBUG? Without the results of these specific experiments, I do not think we can help much. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 07:16 Message: Logged In: NO I guess I'm not making myself very clear. Let me try again... I have a library that completely encapsulates all use of Python.h from my clients. I need to be able to debug this library by compiling it with DEBUG. I do not want to use the debug Python library. Unfortunately, just compiling the Python headers with DEBUG seems to change the object layout and require the use of the debug library. BTW, the sys.path values include my PYTHONPATH values when I link non-debug, and they are the default relative path values when I link with debug. I know the code looks like it doesn't ignore PYTHONPATH (I've looked through getpathp.c too) but something is screwing up sys.path. ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2001-11-08 17:09 Message: Logged In: YES user_id=14198 There is not much I can add here. I understand and appreciate your concerns. However, as Tim and Martin have stated, something is missing from the puzzle. PYTHONPATH is *not* ignored in debug builds. The reason for the _d is 2-fold: * As Martin said, the object layout may change. * The Python core and extensions sometimes share CRT objects - notably file handles, and in some cases memory blocks. It is critical that these come from the same CRT. I could possibly agree that the "_d" convention is dumb and should be dropped. However, it is unclear to me that this bug can be simply summarised as this. This is probably because it is unclear to me how this bug can be summarised at all :) ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 15:28 Message: Logged In: YES user_id=21627 AFAICT, PYTHONPATH is *not* ignored when running a debug build. If you think it is, please run the debug and the nodebug interpreter (i.e. python.exe and python_d.exe), and report the values of sys.path for either case. More likely, the cause of your problems is that the import code looks for different DLL name *in sys.path*. I.e. when you do import foo in a nodebug build, it will try to find foo.pyd or foo.dll. In a debug build, it will try to find foo_d.pyd or foo_d.dll. So if you only have foo.pyd, then importing foo will fail if Python was compiled with _DEBUG. Again, this is by design: foo.pyd might crash the debug interpreter, since the object layout is different. If you merely want to single-step through the Python runtime, without making use of the Py_DEBUG features, you should build the release version of Python, but activate the generation of debug symbols. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 12:03 Message: Logged In: NO You'll have to excuse the editorializing: I've found working with the C API immensely frustrating in a variety of ways, and then to have my empirical observations dismissed... Let me step back and explain where I am right now. I've built the 2.1 debug version of the python library using the source distribution. When doing a final link without debug (regardless of how all my client code was compiled) I use the non-debug version of the Python library. When doing a final link with debug (again, regardless of how all of my client code is compiled) I use the debug version of the python library. To get the latter scenario to work, I edit the registry -- as well as the PYTHONPATH -- before initializing python through the C API. For these two cases, my unit tests run correctly to completion. However, this means that our developers must compile the library I wrote that wraps up the Python C API to match how they are going to link it. I would like to find a way around this constraint so that any combination of compiling and linking my wrapper library works correctly. (Let me also mention that I am aware that you are all volunteers and I do appreciate your efforts. I am hopeful that by resolving issues such as this Python can be made more easy to use in a wider variety of circumstances.) Thanks. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 11:49 Message: Logged In: YES user_id=31435 Anonymous, editorializing doesn't really help. I understand that you want Python to work in a way other than it does, and repeating that isn't necessary. I'm hoping Mark Hammond will comment on that. In the meantime, I'm trying to determine why you're seeing the symptoms you are seeing. I believe you're seeing them, but your analysis doesn't make sense, therefore something is going on you haven't told us about. What *should* happen in your ideal world isn't relevant to determining what's actually happening. Take this one step at a time, please. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:20 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 11:19 Message: Logged In: NO It's not a question of *believing* that PYTHONPATH is ignored: that is the observed behavior! (I am using 2.1.1.) Whether I compile my client code with debug or not should have nothing to do with what Python library I include. (Note that I'm talking about *my* client code, not compiling Python itself.) The Python #includes should not be selecting a library for me, but should let me specify it in my build environment. This is the way virtually every 3rd-party library is set up for both Windows and *nix systems and the way I expected Python to behave. Whether or not Windows defines _DEBUG shouldn't matter either -- I should have independent control. (I even when to the lengths of undeffing DEBUG, _DEBUG, and Py_Debug prior to including any Python headers -- still didn't help.) Keep in mind that Python is not the core of my application. It doesn't control the main loop. In fact I entirely encapsulate the C API to Python so that my clients can define commands without ever having to include Python.h themselves. Python is just one piece of a very large system (and a replacable piece at that) and can't dictate build or compile requirements. I'll check the system libraries, but I'm pretty sure they are the multithreaded versions. (I think I'd be seeing different symptoms if I had the wrong system libraries.) Unfortunately, I can't send you samples due to confidentiality concerns. Rest assured, however, that I am describing the problems accurately. Try it yourself! Uninstall python, just retaining the normal link libraries and dlls and the headers. Now build an application using just the C API. Not as simple as it sounds (or is with competing tools such as Tcl). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 09:48 Message: Logged In: YES user_id=31435 Well, Martin is right, you know: if you're not defining _DEBUG, and not letting MSVC define it either, then neither will pyconfig.h define Py_DEBUG, and it gets harder and harder to believe that what you're reporting is the full truth. Perhaps you could attach a tiny example, along with the MSVC project file, or Makefile, you use. It may well be academic, though, as it's *normal* for _DEBUG to get defined under MSVC in a debug build (and whether or not you add it explicitly -- as Martin implied, you would have had to explicitly remove _DEBUG from the MSVC auto-generated list of preprocesser debug-build #defines). In answer to one of your earlier questions, PYTHONPATH is not ignored after a debug link. I understand that you believe it is, but I expect that's another illusion due to something you haven't told us-- or not discovered --yet. Perhaps you're linking with the wrong system libraries? Python requires the multithreaded libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:33 Message: Logged In: YES user_id=21627 What Python version are you using? You mentioned that you've changed the pragma in config.h; atleast since Python 1.5.2, this macro depends on _DEBUG, not on DEBUG. So if you didn't define _DEBUG, you wouldn't need to change the pragma.Python never changes its behaviour based on whether DEBUG is defined, AFAICT. Please check the project settings for the debug target in your application again. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 09:22 Message: Logged In: NO The only debug-type symbol I am defining when compiling my code is DEBUG. I'm not deliberately defining _DEBUG or Py_Debug. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-08 09:09 Message: Logged In: YES user_id=21627 AFAICT, even on Windows, defining DEBUG does not change anything whatsoever. It is only when you define _DEBUG that things start to change. I'm not sure whether this is a bug or a feature: If you just want to enable debugging in your application, just don't define _DEBUG. Notice that MSVC does that for you unless you tell it not to. AFAICT, _DEBUG is what Microsoft uses in the libraries to indicate "enable all debugging code that you have". So Python enables Py_DEBUG. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-08 08:43 Message: Logged In: YES user_id=31435 Changed Category to Windows, since I'm almost certain that's the missing info here. Assigned to MarkH hoping he has a slick reply. This kind of complaint comes up rarely but regularly, and I'm never sure what to say: since I build Python from source, and always did, our Windows debug scheme is fine by me. But what are pure end-users expected to do? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-08 06:10 Message: Logged In: NO I'm not recompiling the Python libraries. I am only recompiling my personal libraries which use the Python C API. I want to be able to compile my libraries with debug and use the standard Python library. This is not a "flawed" approach. It is extremely common when using 3rd-party libraries to do this. Python should be using a separate #define for its ref-counting debug stuff so that clients can use the standard library and still debug their own stuff. Refer to my second comment regarding the use of PYTHONPATH for another example of non-transparent substitution of the debug and non-debug libraries. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-07 14:11 Message: Logged In: YES user_id=21627 I cannot understand what bug you are reporting. Of the things you report, which of them did you not expect? For example, that Py_RefTotal is undefined when you compile with DEBUG, but link with the no-debug library is not a bug, but by design: those functions are used in every debug module, but available only in the debug library. Your complete compilation approach is flawed: If you compile with pydebug, you absolutely *must* link with the debug library: the object layout of every object will change, so crashes are likely if you link with the wrong library. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-05 09:29 Message: Logged In: NO More info: I've now narrowed it down to the fact that when the final link is done with debug, the PYTHONPATH environment variable is ignored altogether. In my case, I cannot rely on the registry being set up for Python, and the system is resorting to the default relative path names (which, in my case, are of no use). How does the behavior of the (non-debug) python library know to change based on the final link? And why is PYTHONPATH ignored if it is linked in debug? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478339&group_id=5470 From noreply@sourceforge.net Tue Nov 13 13:34:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 05:34:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-481284 ] GetFileSecurity returns wrong SID Message-ID: Bugs item #481284, was opened at 2001-11-13 05:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481284&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Ruben Marquez (rrm1) Assigned to: Tim Peters (tim_one) Summary: GetFileSecurity returns wrong SID Initial Comment: The following code printes PySID:S-1-0x008014000000 for every file on any machine, independent of the real ower of the file: for f in glob.glob("d:/*.*"): try: o = win32security.GetFileSecurity (f,win32security.OWNER_SECURITY_INFORMATION) s = win32security.SID(o) print str(s), except: print "n/a", print " ",f ---------- Interestingly, def prsid(name): import string print string.rjust(name,20), try: sid,box,what=win32security.LookupAccountName (None,name) print str(sid),box,what except: print "oops" Works well, so it doesn't seem to be a problem with PySIDs. Thanks for your help in resolving this. P.S.: (Discussed in http://groups.google.com/groups? hl=en&th=b808d773d7ba0fee) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481284&group_id=5470 From noreply@sourceforge.net Tue Nov 13 16:00:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 08:00:28 -0800 Subject: [Python-bugs-list] [ python-Bugs-481284 ] GetFileSecurity returns wrong SID Message-ID: Bugs item #481284, was opened at 2001-11-13 05:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481284&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Ruben Marquez (rrm1) >Assigned to: Mark Hammond (mhammond) Summary: GetFileSecurity returns wrong SID Initial Comment: The following code printes PySID:S-1-0x008014000000 for every file on any machine, independent of the real ower of the file: for f in glob.glob("d:/*.*"): try: o = win32security.GetFileSecurity (f,win32security.OWNER_SECURITY_INFORMATION) s = win32security.SID(o) print str(s), except: print "n/a", print " ",f ---------- Interestingly, def prsid(name): import string print string.rjust(name,20), try: sid,box,what=win32security.LookupAccountName (None,name) print str(sid),box,what except: print "oops" Works well, so it doesn't seem to be a problem with PySIDs. Thanks for your help in resolving this. P.S.: (Discussed in http://groups.google.com/groups? hl=en&th=b808d773d7ba0fee) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-13 08:00 Message: Logged In: YES user_id=31435 Reassigned to MarkH, as this is in the Win32 extensions. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481284&group_id=5470 From noreply@sourceforge.net Tue Nov 13 17:13:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 09:13:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-481221 ] rfc822.Addrlist class fails on long addr Message-ID: Bugs item #481221, was opened at 2001-11-13 00:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ondrej Palkovsky (ondrap) >Assigned to: Barry Warsaw (bwarsaw) Summary: rfc822.Addrlist class fails on long addr Initial Comment: The Addrlistclass.getaddrlist uses recursion algorithm, unfortunately on some _very_ large address fields it exceeds the maximum recursion. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 From noreply@sourceforge.net Tue Nov 13 17:14:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 09:14:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-480480 ] pydoc #!/build-directory/python Message-ID: Bugs item #480480, was opened at 2001-11-10 16:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480480&group_id=5470 Category: Distutils Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Brian Zhou (bzhou) >Assigned to: A.M. Kuchling (akuchling) Summary: pydoc #!/build-directory/python Initial Comment: Starting from source for example http://www.python.org/ftp/python/2.2/rpms/python2.2- 2.2b1-2.src.rpm "make install" copies and later packages a version of pydoc which has hard-coded #!/build-directory/python This pydoc will be installed on redhat 7.x. The unwanted dependency will also cause the binary RPM installation to fail on redhat 6.2. "make install" should be fixed to use "#!/usr/bin/env python" or simply pick-up Tools/scripts/pydoc. A quick workaround for rpm is to change the python-2.2.spec file: 162,163c162 < sed 's|#!/usr/bin/env python|#!/usr/bin/env python'%{binsuffix}'|' \ < pydoc.old >pydoc --- > sed 's|#!.*python$|#!/usr/bin/env python'% {binsuffix}'|' pydoc.old >pydoc ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-11 07:02 Message: Logged In: YES user_id=21627 Putting in /usr/bin/env is certainly not the right solution. distutils was specifically designed to adjust the path of scripts, to make sure that the Python binary used at run-time is indeed the one used at installation time. Otherwise, the user may break pydoc by putting a different Python installation in front of the path. Instead, distutils should be fixed to assume that the Python interpreter will be in BINDIR. Recategorizing as distutils bug, and raising the priority. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480480&group_id=5470 From noreply@sourceforge.net Tue Nov 13 17:15:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 09:15:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-480096 ] Assign to __debug__ still allowed Message-ID: Bugs item #480096, was opened at 2001-11-09 09:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480096&group_id=5470 Category: Parser/Compiler Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 4 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: Assign to __debug__ still allowed Initial Comment: >From the 2.1c1 NEWS file: """ - After a public outcry, assignment to __debug__ is no longer illegal; instead, a warning is issued. It will become illegal in 2.2. """ That hasn't yet happened. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480096&group_id=5470 From noreply@sourceforge.net Tue Nov 13 17:15:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 09:15:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-480188 ] printing unicode objects Message-ID: Bugs item #480188, was opened at 2001-11-09 13:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: M.-A. Lemburg (lemburg) Summary: printing unicode objects Initial Comment: """ In a print statement using a trailing comma (to suppress printing a newline) printing a unicode object that contains a trailing newline sometimes prints an extra space. If I have a unicode object, u, that contains a trailing newline, then... print str(u), # behaves as expected print u.encode(), # behaves as expected print u, # the newline isn't recognized # and the next print # inserts a space It appears that the implicit form of encoding the unicode object doesn't quite work right with print. The following code illustrates my problem. My output follows. """ import sys print sys.version s = 'This is a test\n' u = unicode(s) print print s, print s, print s, print print str(u), print str(u), print str(u), print print u.encode(), print u.encode(), print u.encode(), print print u, print u, print u, """ My output: 2.1.1 (#1, Aug 28 2001, 19:51:39) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-85)] This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test """ ---------------------------------------------------------------------- Comment By: Daniel Popowich (dpopowich) Date: 2001-11-09 14:57 Message: Logged In: YES user_id=372379 When I submitted this but I didn't realize the form was going to eat my formatting. The last few lines of the output has leading spaces. Run the sample code to see what I mean. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 From noreply@sourceforge.net Tue Nov 13 17:26:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 09:26:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-478421 ] cPickle bug when using imp.load_source() Message-ID: Bugs item #478421, was opened at 2001-11-05 12:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478421&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: Andrew R Gross (sincere2001) Assigned to: Nobody/Anonymous (nobody) Summary: cPickle bug when using imp.load_source() Initial Comment: When a Python 2.1.1 program dynamically creates a module using the 'imp.load_source()' function, and the created module tries to 'import cPickle', the interpreter dies with a SystemError exception: SystemError: _PyImport_FixupExtension: module import_cPickle.cPickle not loaded This does *not* occur when the find/load_module() functions are used. The bug also appears to have been introduced somewhere in 2.x, as 1.5.x works fine with load_source(). Attached is a full explanation of the behavior and some small sample programs to illustrate the bug. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-13 09:26 Message: Logged In: YES user_id=31392 I agree with frangen's explanation. This appears to be a bug in the user code, not in Python. Will reopen if you can provide more info that suggests this really is a Python bug. ---------------------------------------------------------------------- Comment By: Simo Salminen (frangen) Date: 2001-11-12 04:02 Message: Logged In: YES user_id=291461 Your example works fine with my system. However: You are giving imp.load_source bad 'name' paramater. In your example: mod2 = imp.load_source(file, file, open(file)) Change that first paramater to os.path.splitext(file)[0] and it might work. Here is an example program that shows how load_source fails with bad first paramater: egg.py: import imp imp.load_source('spam.py', 'spam.py', open('spam.py')) spam.py: import os.path % python egg.py ImportError: No module named path and now fix that egg.py file: imp.load_source('spam', 'spam.py', open('spam.py')) -> works fine. (tested with activepython 2.1.1) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478421&group_id=5470 From noreply@sourceforge.net Tue Nov 13 17:41:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 09:41:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-481171 ] Hang using files named prn.txt, etc Message-ID: Bugs item #481171, was opened at 2001-11-12 21:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481171&group_id=5470 Category: Windows Group: Platform-specific >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Andrew Bennetts (spiv) Assigned to: Tim Peters (tim_one) Summary: Hang using files named prn.txt, etc Initial Comment: Windows reserves many possible filenames as reserved device names. Examples are prn.xxx, con.xxx, com1.xxx aux.xxx, etc, where xxx can be any extension. Attempts to create/read/write to these files can hang the interpreter, but ideally would just throw an OSError. A partial list of device names is mentioned at: http://cert.uni- stuttgart.de/archive/bugtraq/2001/07/msg00086.html ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-13 09:41 Message: Logged In: YES user_id=31435 Did you read the message you referenced with that URL? I agree with it: """ Conclusion : applications should not filter out DDNs, because they don't fix the problem (basically they make it even worse), the OS patch is better because it fixes *ALL* problems, ... CONCLUSION : patch your OS, and stop whining about so called 'bugs' in applications, you will never be able to completely patch the problem that way. """ They're right: applications cannot fix this; if it's a problem for you, then you should download and apply the MS OS patches linked to from that article. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481171&group_id=5470 From noreply@sourceforge.net Tue Nov 13 17:58:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 09:58:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-478115 ] rfc822.Message.getdate('Date') exception Message-ID: Bugs item #478115, was opened at 2001-11-04 15:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478115&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Piers Lauder (pierslauder) Assigned to: Barry Warsaw (bwarsaw) Summary: rfc822.Message.getdate('Date') exception Initial Comment: rfc822.Message.getdate('Date') will raise an IndexError exception if the message header contains a null Date line. An example Python script that reproduces the exception is attached. As far as I know, this affects all versions of rfc822.py. The error is in rfc822.parsedate_tz which assumes that its string argument is non-empty. The first line should probably be: if not data: return None ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 09:58 Message: Logged In: YES user_id=12800 Fixed in rfc822.py 1.64 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478115&group_id=5470 From noreply@sourceforge.net Tue Nov 13 20:19:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 12:19:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-481221 ] rfc822.Addrlist class fails on long addr Message-ID: Bugs item #481221, was opened at 2001-11-13 00:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 >Category: Python Library Group: None >Status: Pending Resolution: None Priority: 5 Submitted By: Ondrej Palkovsky (ondrap) Assigned to: Barry Warsaw (bwarsaw) Summary: rfc822.Addrlist class fails on long addr Initial Comment: The Addrlistclass.getaddrlist uses recursion algorithm, unfortunately on some _very_ large address fields it exceeds the maximum recursion. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 12:19 Message: Logged In: YES user_id=12800 Please provide attach a test case. It's unlikely that this'll get fixed before Python 2.2 since it fixing it probably requires a rewrite. I wonder if this could use generators now? Can you provide a patch? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 From noreply@sourceforge.net Tue Nov 13 20:52:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 12:52:54 -0800 Subject: [Python-bugs-list] [ python-Bugs-481221 ] rfc822.Addrlist class fails on long addr Message-ID: Bugs item #481221, was opened at 2001-11-13 00:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 Category: Python Library Group: None >Status: Open Resolution: None Priority: 5 Submitted By: Ondrej Palkovsky (ondrap) Assigned to: Barry Warsaw (bwarsaw) Summary: rfc822.Addrlist class fails on long addr Initial Comment: The Addrlistclass.getaddrlist uses recursion algorithm, unfortunately on some _very_ large address fields it exceeds the maximum recursion. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-13 12:52 Message: Logged In: YES user_id=31435 Barry, this very shallow, although it would be nice to have a test case. Currently: . ad = self.getaddress() . if ad: . return ad + self.getaddrlist() . else: return [] Rewrite (untested): . result = [] . while 1: . ad = self.getaddress() . if ad: . result.append(ad) . else: . break . return result The original is also quadratic-time(!) in the # of addresses -- it's obviously braindead Scheme code . ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 12:19 Message: Logged In: YES user_id=12800 Please provide attach a test case. It's unlikely that this'll get fixed before Python 2.2 since it fixing it probably requires a rewrite. I wonder if this could use generators now? Can you provide a patch? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 From noreply@sourceforge.net Tue Nov 13 21:26:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 13:26:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-481221 ] rfc822.Addrlist class fails on long addr Message-ID: Bugs item #481221, was opened at 2001-11-13 00:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Ondrej Palkovsky (ondrap) Assigned to: Barry Warsaw (bwarsaw) Summary: rfc822.Addrlist class fails on long addr Initial Comment: The Addrlistclass.getaddrlist uses recursion algorithm, unfortunately on some _very_ large address fields it exceeds the maximum recursion. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:26 Message: Logged In: YES user_id=12800 Actually result.append() needs to be result.extend(), but I think you're right, it is shallow. I'm commit a fix. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-13 12:52 Message: Logged In: YES user_id=31435 Barry, this very shallow, although it would be nice to have a test case. Currently: . ad = self.getaddress() . if ad: . return ad + self.getaddrlist() . else: return [] Rewrite (untested): . result = [] . while 1: . ad = self.getaddress() . if ad: . result.append(ad) . else: . break . return result The original is also quadratic-time(!) in the # of addresses -- it's obviously braindead Scheme code . ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 12:19 Message: Logged In: YES user_id=12800 Please provide attach a test case. It's unlikely that this'll get fixed before Python 2.2 since it fixing it probably requires a rewrite. I wonder if this could use generators now? Can you provide a patch? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481221&group_id=5470 From noreply@sourceforge.net Tue Nov 13 21:39:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 13:39:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) >Assigned to: Jeremy Hylton (jhylton) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:39 Message: Logged In: YES user_id=12800 Assigning to Jeremy so he'll remember to forward me Jim's email. Jeremy can ping-pong it to Fred if he wants, and then reassign to me after forwarding the email. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Tue Nov 13 22:02:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 14:02:21 -0800 Subject: [Python-bugs-list] [ python-Bugs-474585 ] Deprecate modules Message-ID: Bugs item #474585, was opened at 2001-10-24 12:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open >Resolution: Postponed Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: Deprecate modules Initial Comment: The following modules should be marked as deprecated because the new email package provides all the functionality (using different APIs perhaps, and it steals some implementation from rfc822.py, but still...) rfc822.py mimetools.py MIMEWriter.py Technically mimify.py should be deprecated too, however the email package doesn't provide the run-as-script functionality. That would be easy to provide as a Tools/script, so I still think it should be deprecated. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 14:02 Message: Logged In: YES user_id=12800 Postponing until Python 2.3 ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-10-24 15:06 Message: Logged In: YES user_id=12800 Assigning back to me and re-opening. Not sure it'll be worth it, but this way it'll keep bugging me. :) ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 14:42 Message: Logged In: YES user_id=3066 After discussion with Guido, this appears to be too agressive a deprecation schedule for these modules, especially rfc822. A good first step would be to demonstrate that all uses of these modules can use the email package. Perhaps implementations of these modules should be made using the facilities of the email package? (Don't check them into the library yet; perhaps add them to nondist/sandbox/Lib/ first.) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474585&group_id=5470 From noreply@sourceforge.net Tue Nov 13 22:01:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 14:01:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-478001 ] uninit mem read w/signals Message-ID: Bugs item #478001, was opened at 2001-11-04 06:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 >Category: Python Interpreter Core Group: Python 2.2 Status: Open >Resolution: Rejected Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Tim Peters (tim_one) Summary: uninit mem read w/signals Initial Comment: if the signal()/sigaction() fails, uninitialized memory is returned. the attached patch fixes the problem (didn't test signal() path, only the sigaction() path) Neal ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 14:01 Message: Logged In: YES user_id=12800 Note sigaction() returns 0 on success, -1 on failure, setting the errno code, while signal() returns SIG_ERR on failure. Because PyOS_*sig() attempts to provide a unified interface to the two different functions, we can't support both, and forcing a return of SIG_ERR when sigaction() fails seems to make the most sense. Still, it might be nice to be able to get the more detailed error message when available, and that means we should set the exception (OSError?) and return NULL. And then there's the documentation and backward compatibility problem that Tim mentions. :( I'm not sure what the right answer is. Of the sigaction() errors, it seems that only EINTR should be possible to get from Python (since Python does its own sanity checking on the signal number, etc.). I'm inclined to reject the patch unless it is modified to address the following issues: - return SIG_ERR in all cases when an error occurs - change the call locations to check for this error condition. - preferrably include some test cases (either Python or using the CAPI test framework). - document the new behavior in the C API manual My vote: -1 for changing this in Python 2.2. Rejecting, and reassigning to Tim. Not closed. Changed the bug category. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 11:45 Message: Logged In: YES user_id=31435 Assigned to Barry. Neal's surely correct that we shouldn't be ignoring errors, but returning NULL isn't right either. Should we force return of SIG_ERR then? That's the only *natural* "error return" value, and there's a backward compatibility problem here since we don't document anything about error returns for the {get,set}sig functions, nor do any of the places we call these check for an error return. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 From noreply@sourceforge.net Tue Nov 13 22:27:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 14:27:20 -0800 Subject: [Python-bugs-list] [ python-Bugs-478001 ] uninit mem read w/signals Message-ID: Bugs item #478001, was opened at 2001-11-04 06:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: Rejected Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Tim Peters (tim_one) Summary: uninit mem read w/signals Initial Comment: if the signal()/sigaction() fails, uninitialized memory is returned. the attached patch fixes the problem (didn't test signal() path, only the sigaction() path) Neal ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 14:27 Message: Logged In: YES user_id=12800 Alternatively, I've attached a quick fix that probably doesn't do enough, but might help things for Python 2.2 ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 14:01 Message: Logged In: YES user_id=12800 Note sigaction() returns 0 on success, -1 on failure, setting the errno code, while signal() returns SIG_ERR on failure. Because PyOS_*sig() attempts to provide a unified interface to the two different functions, we can't support both, and forcing a return of SIG_ERR when sigaction() fails seems to make the most sense. Still, it might be nice to be able to get the more detailed error message when available, and that means we should set the exception (OSError?) and return NULL. And then there's the documentation and backward compatibility problem that Tim mentions. :( I'm not sure what the right answer is. Of the sigaction() errors, it seems that only EINTR should be possible to get from Python (since Python does its own sanity checking on the signal number, etc.). I'm inclined to reject the patch unless it is modified to address the following issues: - return SIG_ERR in all cases when an error occurs - change the call locations to check for this error condition. - preferrably include some test cases (either Python or using the CAPI test framework). - document the new behavior in the C API manual My vote: -1 for changing this in Python 2.2. Rejecting, and reassigning to Tim. Not closed. Changed the bug category. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 11:45 Message: Logged In: YES user_id=31435 Assigned to Barry. Neal's surely correct that we shouldn't be ignoring errors, but returning NULL isn't right either. Should we force return of SIG_ERR then? That's the only *natural* "error return" value, and there's a backward compatibility problem here since we don't document anything about error returns for the {get,set}sig functions, nor do any of the places we call these check for an error return. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 From noreply@sourceforge.net Tue Nov 13 23:11:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 15:11:20 -0800 Subject: [Python-bugs-list] [ python-Bugs-472646 ] Cookie.py and : in key Message-ID: Bugs item #472646, was opened at 2001-10-18 21:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472646&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: Cookie.py and : in key Initial Comment: An older version of Cookie.py that I used in Mailman allowed : (colon) in the keys. The version that's in Python right now does not. The question comes down to whether Cookie.py should be a strict interpretation of RFCs 2109 and 2068 (which appears to mandate such keys be enclosed in quotes) or a looser interpretation. On the one hand, it shouldn't allow you to break the standards, but OTOH most browsers allow this, and besides, it's entirely possible that you'll get ill-conformant cookie data. Be liberal in what you accept and strict in what you generate. Here's the thread in python-dev: http://mail.python.org/pipermail/python-dev/2001-October/017722.html Mostly I'm submitting this bug so it doesn't get buried in my inbox. Posted patch is attached. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 15:11 Message: Logged In: YES user_id=12800 I've decided to reject and close this for Python 2.2. I'm not comfortable with making this change so late in the beta cycle. (I've worked around this in Mailman so it isn't of immediate import to me now.) ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-19 10:32 Message: Logged In: YES user_id=3066 This looks good to me. Please check it in and close the bug. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=472646&group_id=5470 From noreply@sourceforge.net Tue Nov 13 23:19:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 15:19:12 -0800 Subject: [Python-bugs-list] [ python-Bugs-478001 ] uninit mem read w/signals Message-ID: Bugs item #478001, was opened at 2001-11-04 06:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed Resolution: Rejected Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Nobody/Anonymous (nobody) Summary: uninit mem read w/signals Initial Comment: if the signal()/sigaction() fails, uninitialized memory is returned. the attached patch fixes the problem (didn't test signal() path, only the sigaction() path) Neal ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-13 15:19 Message: Logged In: YES user_id=31435 Barry checked in an alternative that simply initializes context.sa_handler to SIG_ERR before calling sigaction. This doesn't address the deeper API issues raised, but does address the original "uninitialized memory" bug, so closing this patch. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 14:27 Message: Logged In: YES user_id=12800 Alternatively, I've attached a quick fix that probably doesn't do enough, but might help things for Python 2.2 ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 14:01 Message: Logged In: YES user_id=12800 Note sigaction() returns 0 on success, -1 on failure, setting the errno code, while signal() returns SIG_ERR on failure. Because PyOS_*sig() attempts to provide a unified interface to the two different functions, we can't support both, and forcing a return of SIG_ERR when sigaction() fails seems to make the most sense. Still, it might be nice to be able to get the more detailed error message when available, and that means we should set the exception (OSError?) and return NULL. And then there's the documentation and backward compatibility problem that Tim mentions. :( I'm not sure what the right answer is. Of the sigaction() errors, it seems that only EINTR should be possible to get from Python (since Python does its own sanity checking on the signal number, etc.). I'm inclined to reject the patch unless it is modified to address the following issues: - return SIG_ERR in all cases when an error occurs - change the call locations to check for this error condition. - preferrably include some test cases (either Python or using the CAPI test framework). - document the new behavior in the C API manual My vote: -1 for changing this in Python 2.2. Rejecting, and reassigning to Tim. Not closed. Changed the bug category. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-04 11:45 Message: Logged In: YES user_id=31435 Assigned to Barry. Neal's surely correct that we shouldn't be ignoring errors, but returning NULL isn't right either. Should we force return of SIG_ERR then? That's the only *natural* "error return" value, and there's a backward compatibility problem here since we don't document anything about error returns for the {get,set}sig functions, nor do any of the places we call these check for an error return. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478001&group_id=5470 From noreply@sourceforge.net Wed Nov 14 01:20:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 13 Nov 2001 17:20:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-481171 ] Hang using files named prn.txt, etc Message-ID: Bugs item #481171, was opened at 2001-11-12 21:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481171&group_id=5470 Category: Windows Group: Platform-specific >Status: Open Resolution: Wont Fix >Priority: 2 Submitted By: Andrew Bennetts (spiv) Assigned to: Tim Peters (tim_one) Summary: Hang using files named prn.txt, etc Initial Comment: Windows reserves many possible filenames as reserved device names. Examples are prn.xxx, con.xxx, com1.xxx aux.xxx, etc, where xxx can be any extension. Attempts to create/read/write to these files can hang the interpreter, but ideally would just throw an OSError. A partial list of device names is mentioned at: http://cert.uni- stuttgart.de/archive/bugtraq/2001/07/msg00086.html ---------------------------------------------------------------------- >Comment By: Andrew Bennetts (spiv) Date: 2001-11-13 17:20 Message: Logged In: YES user_id=50945 Yes, I read the message (and most of the thread) :) Apologies for re-opening this bug, but I believe that something can still be done about it. I'm using Windows 2K (SP 2), so those patches don't apply to me. Those patches merely prevent windows crashing on certain types of accesses to devices, e.g. the now well-known C:\NUL\NUL bug. They don't prevent applications from hanging when trying to access prn.txt. Further in the thread starting at that URL, there is some discussion of possible work-arounds for safely opening/creating files. I'm not a Win32 C programmer, so I've no idea about the technical feasibility of these solutions in Python. While I agree that the real problem is in the OS, it seems to be standard behaviour now for apps to workaround it. For example, Textpad (www.textpad.com) will just pop up a dialog saying "You cannot create this file: this name is reserved device name" or something similar. Whether they've just hard-coded a list of known names or if they're doing something more advanced I don't know. This caused a fair bit of confusion for me recently: Python hung inexplicably in splitting a CSV file of stock data into seperate files, and one of the Australian Stock Exchange's stock symbols is "PRN". The script was extremely simple, so I eventually realised what was going on, but I dread an unwary person having to debug this problem in a larger application. If you think this sort of ugliness doesn't belong in Python then feel free to close this bug again, and I won't feel offended :) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-13 09:41 Message: Logged In: YES user_id=31435 Did you read the message you referenced with that URL? I agree with it: """ Conclusion : applications should not filter out DDNs, because they don't fix the problem (basically they make it even worse), the OS patch is better because it fixes *ALL* problems, ... CONCLUSION : patch your OS, and stop whining about so called 'bugs' in applications, you will never be able to completely patch the problem that way. """ They're right: applications cannot fix this; if it's a problem for you, then you should download and apply the MS OS patches linked to from that article. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481171&group_id=5470 From noreply@sourceforge.net Wed Nov 14 14:43:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 06:43:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-479981 ] socket failed then compiled with treads Message-ID: Bugs item #479981, was opened at 2001-11-09 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 Category: Threads Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket failed then compiled with treads Initial Comment: Operating System HP-UX 11.0 (64 Bit) I tried to compile Python 2.1.1 this thread support. Doing so, the tests test_socket and test_asynchat both failed with "(9, 'Bad file number')" The test_socket succeds if I compiled python without threads, so I think it's an thread-problem. The thread-test istself succeeds. I think also, that python is correctly linked against libpthread. The same Problem occurs this Python 2.2.b1. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-14 06:43 Message: Logged In: YES user_id=21627 Python puts a lock around gethostbyname if gethostbyname_r is not available. I don't think that should cause a problem even if gethostbyname is thread-safe. If you want to further analyse this, you could try to put an && !defined(__hpux) around the line #define USE_GETHOSTBYNAME_LOCK If you do, please report whether it works. In that case, it would be good to know whether gethostbyname is thread-safe on *all* HP-UX versions, or just on some. Furthermore, since this wasn't reported before: Is it specific to your installation, or reproducable on all installations? Is it perhaps specific to 64-bit mode? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 02:12 Message: Logged In: NO Maybe the failure has to do with gethostbyname-stuff. according the hpux-manpages theese reentrant interfaces are obsolete: int gethostent_r(struct hostent *result, struct hostent_data *buffer); int gethostbyname_r(const char *name, struct hostent *result, struct hostent_data *buffer); int gethostbyaddr_r(const char *addr, int len, int type, struct hostent *result, struct hostent_data *buffer); int sethostent_r(int stayopen, struct hostent_data *buffer); int endhostent_r(struct hostent_data *buffer); The configure-script has correctly found, that there is only a gethostbyname, but no gethostbyname_r. according to the man-pages the call to gethostbyname IS thread-save: struct hostent *gethostent(void); struct hostent *gethostbyname(const char *name); struct hostent *gethostbyaddr(const char *addr, int len, int type); ... MULTITHREAD USAGE Thread Safe: Yes Cancel Safe: Yes Async-cancel Safe: No Async-signal Safe: No Is it a failure to expect the gethostbyname to be not thread-save ? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 01:29 Message: Logged In: NO the complete compiler line used to link together the python binary: gcc -Wl,-E -Wl,+s -Wl,+b/mosquito/python/lib/python2.1/lib- dynload -o python \ Modules/python.o \ libpython2.1.a -lnsl -ldld -lpthread -lm The compiler I used is gcc 3.01. I tried it out with the cc from HP - the same effect After installing the Kernelpatch PHKL_25475, wich supersedes both PHKL_23842 and PHKL_20942 the failure remains the same ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 16:13 Message: Logged In: YES user_id=21627 Can you please also check whether the patches PHKL_23842 (or equivalent) and/or PHKL_20942. The patch titles are PHKL_23842 s700_800 11.00 signal,threads,spinlock,scheduler,IDS,q3p PHKL_20942 s700_800 11.00 signal, nanosleep, spinlock, scheduler It appears that HP-UX has a number of bugs where signals, forking, and other aspects of the operating functionality stop working in the presence of threads. If you don't have these patches, it would be good if you could try to install them, and report back whether it changes anything. Notice that HP apparently has a number of patches with these titles; they seem to be intended for different machines. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 15:51 Message: Logged In: YES user_id=21627 What compiler are you using? Can you please report the complete compiler line used to link together the python binary? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 From noreply@sourceforge.net Wed Nov 14 16:00:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 08:00:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-481726 ] literal fp pound errors pass silently Message-ID: Bugs item #481726, was opened at 2001-11-14 08:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Grant Griffin (dspguru) Assigned to: Nobody/Anonymous (nobody) Summary: literal fp pound errors pass silently Initial Comment: It looks like anything beyond a pound sign in a floating point literal is allowed, but ignored. Here are a couple of examples: ---------- Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> a=1.#INF >>> a 1.0 ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> a=1.#asdf >>> a 1.0 >>> a=2.#INF >>> a 2.0 ----------- It seems like this notation should either be disallowed totally, or at least be disallowed when it does not result in a valid IEEE magic number (as in the second and third examples). Also, maybe this has an impact on pickling if IEEE magic numbers cannot be remade from their strings, e.g. >>> x=1e300 >>> y=x*x >>> y 1.#INF >>> z=float(str(y)) Traceback (most recent call last): File "", line 1, in ? ValueError: invalid literal for float(): 1.#INF (Note that "float()" does not accept the "#" notation, but the parser does.) thanks, =g2 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 From noreply@sourceforge.net Wed Nov 14 17:47:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 09:47:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-481761 ] test_format fails Message-ID: Bugs item #481761, was opened at 2001-11-14 09:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481761&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: James Richardson (jamesrichardson) Assigned to: Nobody/Anonymous (nobody) Summary: test_format fails Initial Comment: OS: Tru64 4.0F python: 2.2a3 gcc: 2.95.2 Any ideas? viper.htc.com> ./python ./Lib/test/test_format.py '%.1d' % (1,) =? '1' ... yes u'%.1d' % (1,) =? '1' ... yes '%.*d' % (9223372036854775807, 1) works? ... yes u'%.*d' % (9223372036854775807, 1) works? ... yes '%.100d' % (1,) =? '000000000000000000000000000000000000000000000000000 000000000 0000000000000000000000000000000000000001' ... yes u'%.100d' % (1,) =? '000000000000000000000000000000000000000000000000000 00000000 00000000000000000000000000000000000000001' ... yes '%#.117x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 00000000 0000000000000000000000000000000000000000000000000000000 00001' ... yes u'%#.117x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 0000000 0000000000000000000000000000000000000000000000000000000 000001' ... yes '%#.118x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 00000000 0000000000000000000000000000000000000000000000000000000 000001' ... overflow (thi s is fine) u'%#.118x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 0000000 0000000000000000000000000000000000000000000000000000000 0000001' ... overflow (th is is fine) '%f' % (1.0,) =? '1.000000' ... yes u'%f' % (1.0,) =? '1.000000' ... yes '%#.*g' % (109, -3.3333333333333334e+48) works? ... yes u'%#.*g' % (109, -3.3333333333333334e+48) works? ... yes '%#.*g' % (110, -3.3333333333333334e+48) works? ... overflow (this is fine) u'%#.*g' % (110, -3.3333333333333334e+48) works? ... overflow (this is fine) '%#.*g' % (110, -3.3333333333333332e+99) works? ... overflow (this is fine) u'%#.*g' % (110, -3.3333333333333332e+99) works? ... overflow (this is fine) '%12.*f' % (123456, 1.0) works? ... overflow (this is fine) u'%12.*f' % (123456, 1.0) works? ... overflow (this is fine) '%x' % 10L =? 'a' ... yes u'%x' % 10L =? 'a' ... yes '%x' % 100000000000L =? '174876e800' ... yes u'%x' % 100000000000L =? '174876e800' ... yes '%o' % 10L =? '12' ... yes u'%o' % 10L =? '12' ... yes '%o' % 100000000000L =? '1351035564000' ... yes u'%o' % 100000000000L =? '1351035564000' ... yes '%d' % 10L =? '10' ... yes u'%d' % 10L =? '10' ... yes '%d' % 100000000000L =? '100000000000' ... yes u'%d' % 100000000000L =? '100000000000' ... yes '%d' % 123456789012345678901234567890L =? '123456789012345678901234567890' ... y es u'%d' % 123456789012345678901234567890L =? '123456789012345678901234567890' ... yes '%d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' ... yes u'%d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' .. . yes '%5d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' .. . yes u'%5d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' . .. yes '%31d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' . .. yes u'%31d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' ... yes '%32d' % -123456789012345678901234567890L =? ' - 123456789012345678901234567890' ... yes u'%32d' % -123456789012345678901234567890L =? ' - 123456789012345678901234567890' ... yes '%-32d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes u'%-32d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes '%032d' % -123456789012345678901234567890L =? '- 0123456789012345678901234567890' ... yes u'%032d' % -123456789012345678901234567890L =? '- 0123456789012345678901234567890 ' ... yes '%-032d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes u'%-032d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes '%034d' % -123456789012345678901234567890L =? '- 00012345678901234567890123456789 0' ... yes u'%034d' % -123456789012345678901234567890L =? '- 0001234567890123456789012345678 90' ... yes '%034d' % 123456789012345678901234567890L =? '0000123456789012345678901234567890 ' ... yes u'%034d' % 123456789012345678901234567890L =? '000012345678901234567890123456789 0' ... yes '%0+34d' % 123456789012345678901234567890L =? '+00012345678901234567890123456789 0' ... yes u'%0+34d' % 123456789012345678901234567890L =? '+0001234567890123456789012345678 90' ... yes '%+34d' % 123456789012345678901234567890L =? ' +123456789012345678901234567890 ' ... yes u'%+34d' % 123456789012345678901234567890L =? ' +12345678901234567890123456789 0' ... yes '%34d' % 123456789012345678901234567890L =? ' 123456789012345678901234567890' ... yes u'%34d' % 123456789012345678901234567890L =? ' 123456789012345678901234567890 ' ... yes '%.2d' % 123456789012345678901234567890L =? '123456789012345678901234567890' ... yes u'%.2d' % 123456789012345678901234567890L =? '123456789012345678901234567890' .. . yes '%.30d' % 123456789012345678901234567890L =? '123456789012345678901234567890' .. . yes u'%.30d' % 123456789012345678901234567890L =? '123456789012345678901234567890' . .. yes '%.31d' % 123456789012345678901234567890L =? '0123456789012345678901234567890' . .. yes u'%.31d' % 123456789012345678901234567890L =? '0123456789012345678901234567890' ... yes '%32.31d' % 123456789012345678901234567890L =? ' 0123456789012345678901234567890 ' ... yes u'%32.31d' % 123456789012345678901234567890L =? ' 012345678901234567890123456789 0' ... yes '%x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes u'%x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes '%x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes u'%x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes '%5x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes u'%5x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes '%22x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes u'%22x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes '%23x' % -1375488932362216742658885L =? ' - 1234567890abcdef12345' ... yes u'%23x' % -1375488932362216742658885L =? ' - 1234567890abcdef12345' ... yes '%-23x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes u'%-23x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes '%023x' % -1375488932362216742658885L =? '- 01234567890abcdef12345' ... yes u'%023x' % -1375488932362216742658885L =? '- 01234567890abcdef12345' ... yes '%-023x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes u'%-023x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes '%025x' % -1375488932362216742658885L =? '- 0001234567890abcdef12345' ... yes u'%025x' % -1375488932362216742658885L =? '- 0001234567890abcdef12345' ... yes '%025x' % 1375488932362216742658885L =? '00001234567890abcdef12345' ... yes u'%025x' % 1375488932362216742658885L =? '00001234567890abcdef12345' ... yes '%0+25x' % 1375488932362216742658885L =? '+0001234567890abcdef12345' ... yes u'%0+25x' % 1375488932362216742658885L =? '+0001234567890abcdef12345' ... yes '%+25x' % 1375488932362216742658885L =? ' +1234567890abcdef12345' ... yes u'%+25x' % 1375488932362216742658885L =? ' +1234567890abcdef12345' ... yes '%25x' % 1375488932362216742658885L =? ' 1234567890abcdef12345' ... yes u'%25x' % 1375488932362216742658885L =? ' 1234567890abcdef12345' ... yes '%.2x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes u'%.2x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes '%.21x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes u'%.21x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes '%.22x' % 1375488932362216742658885L =? '01234567890abcdef12345' ... yes u'%.22x' % 1375488932362216742658885L =? '01234567890abcdef12345' ... yes '%23.22x' % 1375488932362216742658885L =? ' 01234567890abcdef12345' ... yes u'%23.22x' % 1375488932362216742658885L =? ' 01234567890abcdef12345' ... yes '%-23.22x' % 1375488932362216742658885L =? '01234567890abcdef12345 ' ... yes u'%-23.22x' % 1375488932362216742658885L =? '01234567890abcdef12345 ' ... yes '%X' % 1375488932362216742658885L =? '1234567890ABCDEF12345' ... yes u'%X' % 1375488932362216742658885L =? '1234567890ABCDEF12345' ... yes '%#X' % 1375488932362216742658885L =? '0X1234567890ABCDEF12345' ... yes u'%#X' % 1375488932362216742658885L =? '0X1234567890ABCDEF12345' ... yes '%#x' % 1375488932362216742658885L =? '0x1234567890abcdef12345' ... yes u'%#x' % 1375488932362216742658885L =? '0x1234567890abcdef12345' ... yes '%#x' % -1375488932362216742658885L =? '- 0x1234567890abcdef12345' ... yes u'%#x' % -1375488932362216742658885L =? '- 0x1234567890abcdef12345' ... yes '%#.23x' % -1375488932362216742658885L =? '- 0x001234567890abcdef12345' ... yes u'%#.23x' % -1375488932362216742658885L =? '- 0x001234567890abcdef12345' ... yes '%#+.23x' % 1375488932362216742658885L =? '+0x001234567890abcdef12345' ... yes u'%#+.23x' % 1375488932362216742658885L =? '+0x001234567890abcdef12345' ... yes '%# .23x' % 1375488932362216742658885L =? ' 0x001234567890abcdef12345' ... yes u'%# .23x' % 1375488932362216742658885L =? ' 0x001234567890abcdef12345' ... yes '%#+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes u'%#+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes '%#-+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes u'%#-+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes '%#-+26.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... ye s u'%#-+26.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... y es '%#-+27.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345 ' ... y es u'%#-+27.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345 ' ... yes '%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... ye s u'%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... y es '%#+027.23X' % 1375488932362216742658885L =? '+0X0001234567890ABCDEF12345' ... y es u'%#+027.23X' % 1375488932362216742658885L =? '+0X0001234567890ABCDEF12345' ... yes '%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... ye s u'%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... y es '%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes u'%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes '%o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' .. . yes u'%o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' . .. yes '%5o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' . .. yes u'%5o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' ... yes '%33o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' ... yes u'%33o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' ... yes '%34o' % -12935167030485801517351291832L =? ' - 12345670123456701234567012345670' ... yes u'%34o' % -12935167030485801517351291832L =? ' - 12345670123456701234567012345670 ' ... yes '%-34o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670 ' ... yes u'%-34o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670 ' ... yes '%034o' % -12935167030485801517351291832L =? '- 012345670123456701234567012345670 ' ... yes u'%034o' % -12935167030485801517351291832L =? '- 01234567012345670123456701234567 0' ... yes '%-034o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670 ' ... yes u'%-034o' % -12935167030485801517351291832L =? '- 1234567012345670123456701234567 0 ' ... yes '%036o' % -12935167030485801517351291832L =? '- 000123456701234567012345670123456 70' ... yes u'%036o' % -12935167030485801517351291832L =? '- 00012345670123456701234567012345 670' ... yes '%036o' % 12935167030485801517351291832L =? '00001234567012345670123456701234567 0' ... yes u'%036o' % 12935167030485801517351291832L =? '0000123456701234567012345670123456 70' ... yes '%0+36o' % 12935167030485801517351291832L =? '+000123456701234567012345670123456 70' ... yes u'%0+36o' % 12935167030485801517351291832L =? '+00012345670123456701234567012345 670' ... yes '%+36o' % 12935167030485801517351291832L =? ' +1234567012345670123456701234567 0' ... yes u'%+36o' % 12935167030485801517351291832L =? ' +123456701234567012345670123456 70' ... yes '%36o' % 12935167030485801517351291832L =? ' 12345670123456701234567012345670 ' ... yes u'%36o' % 12935167030485801517351291832L =? ' 1234567012345670123456701234567 0' ... yes '%.2o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' .. . yes u'%.2o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' . .. yes '%.32o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' . .. yes u'%.32o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes '%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes '%34.33o' % 12935167030485801517351291832L =? ' 01234567012345670123456701234567 0' ... yes u'%34.33o' % 12935167030485801517351291832L =? ' 0123456701234567012345670123456 70' ... yes '%-34.33o' % 12935167030485801517351291832L =? '01234567012345670123456701234567 0 ' ... yes u'%-34.33o' % 12935167030485801517351291832L =? '0123456701234567012345670123456 70 ' ... yes '%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes u'%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes '%#o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' .. . yes u'%#o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' . .. yes '%#o' % -12935167030485801517351291832L =? '- 012345670123456701234567012345670' ... yes u'%#o' % -12935167030485801517351291832L =? '- 012345670123456701234567012345670' ... yes '%#.34o' % -12935167030485801517351291832L =? '- 00123456701234567012345670123456 70' ... yes u'%#.34o' % -12935167030485801517351291832L =? '- 0012345670123456701234567012345 670' ... yes '%#+.34o' % 12935167030485801517351291832L =? '+00123456701234567012345670123456 70' ... yes u'%#+.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012345 670' ... yes '%# .34o' % 12935167030485801517351291832L =? ' 00123456701234567012345670123456 70' ... yes u'%# .34o' % 12935167030485801517351291832L =? ' 0012345670123456701234567012345 670' ... yes '%#+.34o' % 12935167030485801517351291832L =? '+00123456701234567012345670123456 70' ... yes u'%#+.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012345 670' ... yes '%#-+.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012345 670' ... yes u'%#-+.34o' % 12935167030485801517351291832L =? '+001234567012345670123456701234 5670' ... yes '%#-+37.34o' % 12935167030485801517351291832L =? '+00123456701234567012345670123 45670 ' ... yes u'%#-+37.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012 345670 ' ... yes '%#+37.34o' % 12935167030485801517351291832L =? ' +0012345670123456701234567012 345670' ... yes u'%#+37.34o' % 12935167030485801517351291832L =? ' +001234567012345670123456701 2345670' ... yes '%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes '%#.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%#.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670 ' ... yes '%#.32o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%#.32o' % 12935167030485801517351291832L =? '012345670123456701234567012345670 ' ... yes '%034.33o' % 12935167030485801517351291832L =? '00123456701234567012345670123456 70' ... yes u'%034.33o' % 12935167030485801517351291832L =? '0012345670123456701234567012345 670' ... yes '%0#34.33o' % 12935167030485801517351291832L =? '0012345670123456701234567012345 670' ... yes u'%0#34.33o' % 12935167030485801517351291832L =? '001234567012345670123456701234 5670' ... yes '%d' % 42 =? '42' ... yes u'%d' % 42 =? '42' ... yes '%d' % -42 =? '-42' ... yes u'%d' % -42 =? '-42' ... yes '%d' % 42L =? '42' ... yes u'%d' % 42L =? '42' ... yes '%d' % -42L =? '-42' ... yes u'%d' % -42L =? '-42' ... yes '%#x' % 1 =? '0x1' ... yes u'%#x' % 1 =? '0x1' ... yes '%#x' % 1L =? '0x1' ... yes u'%#x' % 1L =? '0x1' ... yes '%#X' % 1 =? '0X1' ... yes u'%#X' % 1 =? '0X1' ... yes '%#X' % 1L =? '0X1' ... yes u'%#X' % 1L =? '0X1' ... yes '%#o' % 1 =? '01' ... yes u'%#o' % 1 =? '01' ... yes '%#o' % 1L =? '01' ... yes u'%#o' % 1L =? '01' ... yes '%#o' % 0 =? '0' ... no '%#o' % 0 == '00' != '0' u'%#o' % 0 =? '0' ... no u'%#o' % 0 == u'00' != '0' '%#o' % 0L =? '0' ... yes u'%#o' % 0L =? '0' ... yes '%o' % 0 =? '0' ... yes u'%o' % 0 =? '0' ... yes '%o' % 0L =? '0' ... yes u'%o' % 0L =? '0' ... yes '%d' % 0 =? '0' ... yes u'%d' % 0 =? '0' ... yes '%d' % 0L =? '0' ... yes u'%d' % 0L =? '0' ... yes '%#x' % 0 =? '0x0' ... yes u'%#x' % 0 =? '0x0' ... yes '%#x' % 0L =? '0x0' ... yes u'%#x' % 0L =? '0x0' ... yes '%#X' % 0 =? '0X0' ... yes u'%#X' % 0 =? '0X0' ... yes '%#X' % 0L =? '0X0' ... yes u'%#X' % 0L =? '0X0' ... yes '%x' % 66 =? '42' ... yes u'%x' % 66 =? '42' ... yes '%x' % 66L =? '42' ... yes u'%x' % 66L =? '42' ... yes '%x' % -66L =? '-42' ... yes u'%x' % -66L =? '-42' ... yes '%o' % 34 =? '42' ... yes u'%o' % 34 =? '42' ... yes '%o' % 34L =? '42' ... yes u'%o' % 34L =? '42' ... yes '%o' % -34L =? '-42' ... yes u'%o' % -34L =? '-42' ... yes Testing exceptions 'abc %a' % 1 works? ... yes u'abc %\u3000' % 1 works? ... yes viper.htc.com> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481761&group_id=5470 From noreply@sourceforge.net Wed Nov 14 18:07:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 10:07:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-481726 ] literal fp pound errors pass silently Message-ID: Bugs item #481726, was opened at 2001-11-14 08:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Grant Griffin (dspguru) Assigned to: Nobody/Anonymous (nobody) Summary: literal fp pound errors pass silently Initial Comment: It looks like anything beyond a pound sign in a floating point literal is allowed, but ignored. Here are a couple of examples: ---------- Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> a=1.#INF >>> a 1.0 ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> a=1.#asdf >>> a 1.0 >>> a=2.#INF >>> a 2.0 ----------- It seems like this notation should either be disallowed totally, or at least be disallowed when it does not result in a valid IEEE magic number (as in the second and third examples). Also, maybe this has an impact on pickling if IEEE magic numbers cannot be remade from their strings, e.g. >>> x=1e300 >>> y=x*x >>> y 1.#INF >>> z=float(str(y)) Traceback (most recent call last): File "", line 1, in ? ValueError: invalid literal for float(): 1.#INF (Note that "float()" does not accept the "#" notation, but the parser does.) thanks, =g2 ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-11-14 10:07 Message: Logged In: YES user_id=6656 You haven't forgotten the fact that # is the comment character in Python, have you? This makes it "tricky" to implement reading 1.#INF as anything other than 1. ! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 From noreply@sourceforge.net Wed Nov 14 18:12:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 10:12:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-481761 ] test_format fails Message-ID: Bugs item #481761, was opened at 2001-11-14 09:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481761&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: James Richardson (jamesrichardson) >Assigned to: Michael Hudson (mwh) Summary: test_format fails Initial Comment: OS: Tru64 4.0F python: 2.2a3 gcc: 2.95.2 Any ideas? viper.htc.com> ./python ./Lib/test/test_format.py '%.1d' % (1,) =? '1' ... yes u'%.1d' % (1,) =? '1' ... yes '%.*d' % (9223372036854775807, 1) works? ... yes u'%.*d' % (9223372036854775807, 1) works? ... yes '%.100d' % (1,) =? '000000000000000000000000000000000000000000000000000 000000000 0000000000000000000000000000000000000001' ... yes u'%.100d' % (1,) =? '000000000000000000000000000000000000000000000000000 00000000 00000000000000000000000000000000000000001' ... yes '%#.117x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 00000000 0000000000000000000000000000000000000000000000000000000 00001' ... yes u'%#.117x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 0000000 0000000000000000000000000000000000000000000000000000000 000001' ... yes '%#.118x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 00000000 0000000000000000000000000000000000000000000000000000000 000001' ... overflow (thi s is fine) u'%#.118x' % (1,) =? '0x0000000000000000000000000000000000000000000000000 0000000 0000000000000000000000000000000000000000000000000000000 0000001' ... overflow (th is is fine) '%f' % (1.0,) =? '1.000000' ... yes u'%f' % (1.0,) =? '1.000000' ... yes '%#.*g' % (109, -3.3333333333333334e+48) works? ... yes u'%#.*g' % (109, -3.3333333333333334e+48) works? ... yes '%#.*g' % (110, -3.3333333333333334e+48) works? ... overflow (this is fine) u'%#.*g' % (110, -3.3333333333333334e+48) works? ... overflow (this is fine) '%#.*g' % (110, -3.3333333333333332e+99) works? ... overflow (this is fine) u'%#.*g' % (110, -3.3333333333333332e+99) works? ... overflow (this is fine) '%12.*f' % (123456, 1.0) works? ... overflow (this is fine) u'%12.*f' % (123456, 1.0) works? ... overflow (this is fine) '%x' % 10L =? 'a' ... yes u'%x' % 10L =? 'a' ... yes '%x' % 100000000000L =? '174876e800' ... yes u'%x' % 100000000000L =? '174876e800' ... yes '%o' % 10L =? '12' ... yes u'%o' % 10L =? '12' ... yes '%o' % 100000000000L =? '1351035564000' ... yes u'%o' % 100000000000L =? '1351035564000' ... yes '%d' % 10L =? '10' ... yes u'%d' % 10L =? '10' ... yes '%d' % 100000000000L =? '100000000000' ... yes u'%d' % 100000000000L =? '100000000000' ... yes '%d' % 123456789012345678901234567890L =? '123456789012345678901234567890' ... y es u'%d' % 123456789012345678901234567890L =? '123456789012345678901234567890' ... yes '%d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' ... yes u'%d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' .. . yes '%5d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' .. . yes u'%5d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' . .. yes '%31d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' . .. yes u'%31d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890' ... yes '%32d' % -123456789012345678901234567890L =? ' - 123456789012345678901234567890' ... yes u'%32d' % -123456789012345678901234567890L =? ' - 123456789012345678901234567890' ... yes '%-32d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes u'%-32d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes '%032d' % -123456789012345678901234567890L =? '- 0123456789012345678901234567890' ... yes u'%032d' % -123456789012345678901234567890L =? '- 0123456789012345678901234567890 ' ... yes '%-032d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes u'%-032d' % -123456789012345678901234567890L =? '- 123456789012345678901234567890 ' ... yes '%034d' % -123456789012345678901234567890L =? '- 00012345678901234567890123456789 0' ... yes u'%034d' % -123456789012345678901234567890L =? '- 0001234567890123456789012345678 90' ... yes '%034d' % 123456789012345678901234567890L =? '0000123456789012345678901234567890 ' ... yes u'%034d' % 123456789012345678901234567890L =? '000012345678901234567890123456789 0' ... yes '%0+34d' % 123456789012345678901234567890L =? '+00012345678901234567890123456789 0' ... yes u'%0+34d' % 123456789012345678901234567890L =? '+0001234567890123456789012345678 90' ... yes '%+34d' % 123456789012345678901234567890L =? ' +123456789012345678901234567890 ' ... yes u'%+34d' % 123456789012345678901234567890L =? ' +12345678901234567890123456789 0' ... yes '%34d' % 123456789012345678901234567890L =? ' 123456789012345678901234567890' ... yes u'%34d' % 123456789012345678901234567890L =? ' 123456789012345678901234567890 ' ... yes '%.2d' % 123456789012345678901234567890L =? '123456789012345678901234567890' ... yes u'%.2d' % 123456789012345678901234567890L =? '123456789012345678901234567890' .. . yes '%.30d' % 123456789012345678901234567890L =? '123456789012345678901234567890' .. . yes u'%.30d' % 123456789012345678901234567890L =? '123456789012345678901234567890' . .. yes '%.31d' % 123456789012345678901234567890L =? '0123456789012345678901234567890' . .. yes u'%.31d' % 123456789012345678901234567890L =? '0123456789012345678901234567890' ... yes '%32.31d' % 123456789012345678901234567890L =? ' 0123456789012345678901234567890 ' ... yes u'%32.31d' % 123456789012345678901234567890L =? ' 012345678901234567890123456789 0' ... yes '%x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes u'%x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes '%x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes u'%x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes '%5x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes u'%5x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes '%22x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes u'%22x' % -1375488932362216742658885L =? '- 1234567890abcdef12345' ... yes '%23x' % -1375488932362216742658885L =? ' - 1234567890abcdef12345' ... yes u'%23x' % -1375488932362216742658885L =? ' - 1234567890abcdef12345' ... yes '%-23x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes u'%-23x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes '%023x' % -1375488932362216742658885L =? '- 01234567890abcdef12345' ... yes u'%023x' % -1375488932362216742658885L =? '- 01234567890abcdef12345' ... yes '%-023x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes u'%-023x' % -1375488932362216742658885L =? '- 1234567890abcdef12345 ' ... yes '%025x' % -1375488932362216742658885L =? '- 0001234567890abcdef12345' ... yes u'%025x' % -1375488932362216742658885L =? '- 0001234567890abcdef12345' ... yes '%025x' % 1375488932362216742658885L =? '00001234567890abcdef12345' ... yes u'%025x' % 1375488932362216742658885L =? '00001234567890abcdef12345' ... yes '%0+25x' % 1375488932362216742658885L =? '+0001234567890abcdef12345' ... yes u'%0+25x' % 1375488932362216742658885L =? '+0001234567890abcdef12345' ... yes '%+25x' % 1375488932362216742658885L =? ' +1234567890abcdef12345' ... yes u'%+25x' % 1375488932362216742658885L =? ' +1234567890abcdef12345' ... yes '%25x' % 1375488932362216742658885L =? ' 1234567890abcdef12345' ... yes u'%25x' % 1375488932362216742658885L =? ' 1234567890abcdef12345' ... yes '%.2x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes u'%.2x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes '%.21x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes u'%.21x' % 1375488932362216742658885L =? '1234567890abcdef12345' ... yes '%.22x' % 1375488932362216742658885L =? '01234567890abcdef12345' ... yes u'%.22x' % 1375488932362216742658885L =? '01234567890abcdef12345' ... yes '%23.22x' % 1375488932362216742658885L =? ' 01234567890abcdef12345' ... yes u'%23.22x' % 1375488932362216742658885L =? ' 01234567890abcdef12345' ... yes '%-23.22x' % 1375488932362216742658885L =? '01234567890abcdef12345 ' ... yes u'%-23.22x' % 1375488932362216742658885L =? '01234567890abcdef12345 ' ... yes '%X' % 1375488932362216742658885L =? '1234567890ABCDEF12345' ... yes u'%X' % 1375488932362216742658885L =? '1234567890ABCDEF12345' ... yes '%#X' % 1375488932362216742658885L =? '0X1234567890ABCDEF12345' ... yes u'%#X' % 1375488932362216742658885L =? '0X1234567890ABCDEF12345' ... yes '%#x' % 1375488932362216742658885L =? '0x1234567890abcdef12345' ... yes u'%#x' % 1375488932362216742658885L =? '0x1234567890abcdef12345' ... yes '%#x' % -1375488932362216742658885L =? '- 0x1234567890abcdef12345' ... yes u'%#x' % -1375488932362216742658885L =? '- 0x1234567890abcdef12345' ... yes '%#.23x' % -1375488932362216742658885L =? '- 0x001234567890abcdef12345' ... yes u'%#.23x' % -1375488932362216742658885L =? '- 0x001234567890abcdef12345' ... yes '%#+.23x' % 1375488932362216742658885L =? '+0x001234567890abcdef12345' ... yes u'%#+.23x' % 1375488932362216742658885L =? '+0x001234567890abcdef12345' ... yes '%# .23x' % 1375488932362216742658885L =? ' 0x001234567890abcdef12345' ... yes u'%# .23x' % 1375488932362216742658885L =? ' 0x001234567890abcdef12345' ... yes '%#+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes u'%#+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes '%#-+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes u'%#-+.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... yes '%#-+26.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... ye s u'%#-+26.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345' ... y es '%#-+27.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345 ' ... y es u'%#-+27.23X' % 1375488932362216742658885L =? '+0X001234567890ABCDEF12345 ' ... yes '%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... ye s u'%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... y es '%#+027.23X' % 1375488932362216742658885L =? '+0X0001234567890ABCDEF12345' ... y es u'%#+027.23X' % 1375488932362216742658885L =? '+0X0001234567890ABCDEF12345' ... yes '%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... ye s u'%#+27.23X' % 1375488932362216742658885L =? ' +0X001234567890ABCDEF12345' ... y es '%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes u'%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes '%o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' .. . yes u'%o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' . .. yes '%5o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' . .. yes u'%5o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' ... yes '%33o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' ... yes u'%33o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670' ... yes '%34o' % -12935167030485801517351291832L =? ' - 12345670123456701234567012345670' ... yes u'%34o' % -12935167030485801517351291832L =? ' - 12345670123456701234567012345670 ' ... yes '%-34o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670 ' ... yes u'%-34o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670 ' ... yes '%034o' % -12935167030485801517351291832L =? '- 012345670123456701234567012345670 ' ... yes u'%034o' % -12935167030485801517351291832L =? '- 01234567012345670123456701234567 0' ... yes '%-034o' % -12935167030485801517351291832L =? '- 12345670123456701234567012345670 ' ... yes u'%-034o' % -12935167030485801517351291832L =? '- 1234567012345670123456701234567 0 ' ... yes '%036o' % -12935167030485801517351291832L =? '- 000123456701234567012345670123456 70' ... yes u'%036o' % -12935167030485801517351291832L =? '- 00012345670123456701234567012345 670' ... yes '%036o' % 12935167030485801517351291832L =? '00001234567012345670123456701234567 0' ... yes u'%036o' % 12935167030485801517351291832L =? '0000123456701234567012345670123456 70' ... yes '%0+36o' % 12935167030485801517351291832L =? '+000123456701234567012345670123456 70' ... yes u'%0+36o' % 12935167030485801517351291832L =? '+00012345670123456701234567012345 670' ... yes '%+36o' % 12935167030485801517351291832L =? ' +1234567012345670123456701234567 0' ... yes u'%+36o' % 12935167030485801517351291832L =? ' +123456701234567012345670123456 70' ... yes '%36o' % 12935167030485801517351291832L =? ' 12345670123456701234567012345670 ' ... yes u'%36o' % 12935167030485801517351291832L =? ' 1234567012345670123456701234567 0' ... yes '%.2o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' .. . yes u'%.2o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' . .. yes '%.32o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' . .. yes u'%.32o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes '%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes '%34.33o' % 12935167030485801517351291832L =? ' 01234567012345670123456701234567 0' ... yes u'%34.33o' % 12935167030485801517351291832L =? ' 0123456701234567012345670123456 70' ... yes '%-34.33o' % 12935167030485801517351291832L =? '01234567012345670123456701234567 0 ' ... yes u'%-34.33o' % 12935167030485801517351291832L =? '0123456701234567012345670123456 70 ' ... yes '%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes u'%o' % 12935167030485801517351291832L =? '12345670123456701234567012345670' ... yes '%#o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' .. . yes u'%#o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' . .. yes '%#o' % -12935167030485801517351291832L =? '- 012345670123456701234567012345670' ... yes u'%#o' % -12935167030485801517351291832L =? '- 012345670123456701234567012345670' ... yes '%#.34o' % -12935167030485801517351291832L =? '- 00123456701234567012345670123456 70' ... yes u'%#.34o' % -12935167030485801517351291832L =? '- 0012345670123456701234567012345 670' ... yes '%#+.34o' % 12935167030485801517351291832L =? '+00123456701234567012345670123456 70' ... yes u'%#+.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012345 670' ... yes '%# .34o' % 12935167030485801517351291832L =? ' 00123456701234567012345670123456 70' ... yes u'%# .34o' % 12935167030485801517351291832L =? ' 0012345670123456701234567012345 670' ... yes '%#+.34o' % 12935167030485801517351291832L =? '+00123456701234567012345670123456 70' ... yes u'%#+.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012345 670' ... yes '%#-+.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012345 670' ... yes u'%#-+.34o' % 12935167030485801517351291832L =? '+001234567012345670123456701234 5670' ... yes '%#-+37.34o' % 12935167030485801517351291832L =? '+00123456701234567012345670123 45670 ' ... yes u'%#-+37.34o' % 12935167030485801517351291832L =? '+0012345670123456701234567012 345670 ' ... yes '%#+37.34o' % 12935167030485801517351291832L =? ' +0012345670123456701234567012 345670' ... yes u'%#+37.34o' % 12935167030485801517351291832L =? ' +001234567012345670123456701 2345670' ... yes '%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes '%#.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%#.33o' % 12935167030485801517351291832L =? '012345670123456701234567012345670 ' ... yes '%#.32o' % 12935167030485801517351291832L =? '012345670123456701234567012345670' ... yes u'%#.32o' % 12935167030485801517351291832L =? '012345670123456701234567012345670 ' ... yes '%034.33o' % 12935167030485801517351291832L =? '00123456701234567012345670123456 70' ... yes u'%034.33o' % 12935167030485801517351291832L =? '0012345670123456701234567012345 670' ... yes '%0#34.33o' % 12935167030485801517351291832L =? '0012345670123456701234567012345 670' ... yes u'%0#34.33o' % 12935167030485801517351291832L =? '001234567012345670123456701234 5670' ... yes '%d' % 42 =? '42' ... yes u'%d' % 42 =? '42' ... yes '%d' % -42 =? '-42' ... yes u'%d' % -42 =? '-42' ... yes '%d' % 42L =? '42' ... yes u'%d' % 42L =? '42' ... yes '%d' % -42L =? '-42' ... yes u'%d' % -42L =? '-42' ... yes '%#x' % 1 =? '0x1' ... yes u'%#x' % 1 =? '0x1' ... yes '%#x' % 1L =? '0x1' ... yes u'%#x' % 1L =? '0x1' ... yes '%#X' % 1 =? '0X1' ... yes u'%#X' % 1 =? '0X1' ... yes '%#X' % 1L =? '0X1' ... yes u'%#X' % 1L =? '0X1' ... yes '%#o' % 1 =? '01' ... yes u'%#o' % 1 =? '01' ... yes '%#o' % 1L =? '01' ... yes u'%#o' % 1L =? '01' ... yes '%#o' % 0 =? '0' ... no '%#o' % 0 == '00' != '0' u'%#o' % 0 =? '0' ... no u'%#o' % 0 == u'00' != '0' '%#o' % 0L =? '0' ... yes u'%#o' % 0L =? '0' ... yes '%o' % 0 =? '0' ... yes u'%o' % 0 =? '0' ... yes '%o' % 0L =? '0' ... yes u'%o' % 0L =? '0' ... yes '%d' % 0 =? '0' ... yes u'%d' % 0 =? '0' ... yes '%d' % 0L =? '0' ... yes u'%d' % 0L =? '0' ... yes '%#x' % 0 =? '0x0' ... yes u'%#x' % 0 =? '0x0' ... yes '%#x' % 0L =? '0x0' ... yes u'%#x' % 0L =? '0x0' ... yes '%#X' % 0 =? '0X0' ... yes u'%#X' % 0 =? '0X0' ... yes '%#X' % 0L =? '0X0' ... yes u'%#X' % 0L =? '0X0' ... yes '%x' % 66 =? '42' ... yes u'%x' % 66 =? '42' ... yes '%x' % 66L =? '42' ... yes u'%x' % 66L =? '42' ... yes '%x' % -66L =? '-42' ... yes u'%x' % -66L =? '-42' ... yes '%o' % 34 =? '42' ... yes u'%o' % 34 =? '42' ... yes '%o' % 34L =? '42' ... yes u'%o' % 34L =? '42' ... yes '%o' % -34L =? '-42' ... yes u'%o' % -34L =? '-42' ... yes Testing exceptions 'abc %a' % 1 works? ... yes u'abc %\u3000' % 1 works? ... yes viper.htc.com> ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-11-14 10:12 Message: Logged In: YES user_id=6656 Dup of #422339. Executive summary: your libc is bust. Complain to Digital^WCompaq^WHP^Wwhoever. You could have trimmed that a bit, you know. It was hard to find the failing bit (even though I knew what it was probably going to be). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481761&group_id=5470 From noreply@sourceforge.net Wed Nov 14 18:17:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 10:17:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-481726 ] literal fp pound errors pass silently Message-ID: Bugs item #481726, was opened at 2001-11-14 08:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 Category: None >Group: Not a Bug >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Grant Griffin (dspguru) Assigned to: Nobody/Anonymous (nobody) Summary: literal fp pound errors pass silently Initial Comment: It looks like anything beyond a pound sign in a floating point literal is allowed, but ignored. Here are a couple of examples: ---------- Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> a=1.#INF >>> a 1.0 ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> a=1.#asdf >>> a 1.0 >>> a=2.#INF >>> a 2.0 ----------- It seems like this notation should either be disallowed totally, or at least be disallowed when it does not result in a valid IEEE magic number (as in the second and third examples). Also, maybe this has an impact on pickling if IEEE magic numbers cannot be remade from their strings, e.g. >>> x=1e300 >>> y=x*x >>> y 1.#INF >>> z=float(str(y)) Traceback (most recent call last): File "", line 1, in ? ValueError: invalid literal for float(): 1.#INF (Note that "float()" does not accept the "#" notation, but the parser does.) thanks, =g2 ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-14 10:17 Message: Logged In: YES user_id=31435 Not a bug: as Michael said, "#" is Python's comment character. See the Reference Manual for Python's grammar for float literals. BTW, I expect you're running on Windows, as I don't know of any other system that spells infinity as "1.#INF". float ("1.#INF") fails on Windows because Microsoft's C floating input routines don't accept it (although Microsoft's C floating output routines produce it). Other platforms are much better behaved in this respect (they can read what they write). Beyond all that, Python has no knowledge of 754 gimmicks. Repairing that is a feature request in PEP 42; that's a major project (IMO), but with no volunteers in sight. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-14 10:07 Message: Logged In: YES user_id=6656 You haven't forgotten the fact that # is the comment character in Python, have you? This makes it "tricky" to implement reading 1.#INF as anything other than 1. ! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 From noreply@sourceforge.net Wed Nov 14 18:35:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 10:35:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-481726 ] literal fp pound errors pass silently Message-ID: Bugs item #481726, was opened at 2001-11-14 08:00 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 Category: None Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: Grant Griffin (dspguru) Assigned to: Nobody/Anonymous (nobody) Summary: literal fp pound errors pass silently Initial Comment: It looks like anything beyond a pound sign in a floating point literal is allowed, but ignored. Here are a couple of examples: ---------- Python 2.2b1 (#25, Oct 19 2001, 11:44:52) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> a=1.#INF >>> a 1.0 ActivePython 2.1.1, build 212 (ActiveState) Python 2.1.1 (#20, Jul 26 2001, 11:38:51) [MSC 32 bit (Intel)] on win32 Type "copyright", "credits" or "license" for more information. >>> a=1.#asdf >>> a 1.0 >>> a=2.#INF >>> a 2.0 ----------- It seems like this notation should either be disallowed totally, or at least be disallowed when it does not result in a valid IEEE magic number (as in the second and third examples). Also, maybe this has an impact on pickling if IEEE magic numbers cannot be remade from their strings, e.g. >>> x=1e300 >>> y=x*x >>> y 1.#INF >>> z=float(str(y)) Traceback (most recent call last): File "", line 1, in ? ValueError: invalid literal for float(): 1.#INF (Note that "float()" does not accept the "#" notation, but the parser does.) thanks, =g2 ---------------------------------------------------------------------- >Comment By: Grant Griffin (dspguru) Date: 2001-11-14 10:35 Message: Logged In: YES user_id=70844 Oops! Sorry about that guys! I guess I forgot about the comment thing. (Must be doing too much C++ lately .) =g2 ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 10:17 Message: Logged In: YES user_id=31435 Not a bug: as Michael said, "#" is Python's comment character. See the Reference Manual for Python's grammar for float literals. BTW, I expect you're running on Windows, as I don't know of any other system that spells infinity as "1.#INF". float ("1.#INF") fails on Windows because Microsoft's C floating input routines don't accept it (although Microsoft's C floating output routines produce it). Other platforms are much better behaved in this respect (they can read what they write). Beyond all that, Python has no knowledge of 754 gimmicks. Repairing that is a feature request in PEP 42; that's a major project (IMO), but with no volunteers in sight. ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-14 10:07 Message: Logged In: YES user_id=6656 You haven't forgotten the fact that # is the comment character in Python, have you? This makes it "tricky" to implement reading 1.#INF as anything other than 1. ! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481726&group_id=5470 From noreply@sourceforge.net Wed Nov 14 19:11:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 11:11:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) >Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:39 Message: Logged In: YES user_id=12800 Assigning to Jeremy so he'll remember to forward me Jim's email. Jeremy can ping-pong it to Fred if he wants, and then reassign to me after forwarding the email. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Wed Nov 14 22:43:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 14:43:21 -0800 Subject: [Python-bugs-list] [ python-Bugs-481882 ] pickle/cPickle can raise SystemError Message-ID: Bugs item #481882, was opened at 2001-11-14 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 6 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: pickle/cPickle can raise SystemError Initial Comment: pickle and cPickle can raise a SystemError when certain things fail during unpickling. For example, when attempting to unpickle a missing named reference (i.e. a class in a non-importable module, or a missing function in an existing module). See below. SystemError clearly is not the right exception to raise here, given the description of the error in the exceptions module (it tells you to contact your Python maintainer!). pickle and cPickle should probably simply propagate the original exception without masking it to SystemError. ----- Python 2.2b1+ (#3, Nov 13 2001, 18:06:11) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> pickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? File "/home/barry/projects/python/Lib/pickle.py", line 979, in loads return Unpickler(file).load() File "/home/barry/projects/python/Lib/pickle.py", line 588, in load dispatch[key](self) File "/home/barry/projects/python/Lib/pickle.py", line 805, in load_global klass = self.find_class(module, name) File "/home/barry/projects/python/Lib/pickle.py", line 815, in find_class raise SystemError, \ SystemError: Failed to import class oops from module __builtin__ >>> import cPickle >>> cPickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? SystemError: Failed to import class oops from module __builtin__ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 From noreply@sourceforge.net Wed Nov 14 22:46:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 14:46:17 -0800 Subject: [Python-bugs-list] [ python-Bugs-481882 ] pickle/cPickle can raise SystemError Message-ID: Bugs item #481882, was opened at 2001-11-14 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 6 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: pickle/cPickle can raise SystemError Initial Comment: pickle and cPickle can raise a SystemError when certain things fail during unpickling. For example, when attempting to unpickle a missing named reference (i.e. a class in a non-importable module, or a missing function in an existing module). See below. SystemError clearly is not the right exception to raise here, given the description of the error in the exceptions module (it tells you to contact your Python maintainer!). pickle and cPickle should probably simply propagate the original exception without masking it to SystemError. ----- Python 2.2b1+ (#3, Nov 13 2001, 18:06:11) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> pickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? File "/home/barry/projects/python/Lib/pickle.py", line 979, in loads return Unpickler(file).load() File "/home/barry/projects/python/Lib/pickle.py", line 588, in load dispatch[key](self) File "/home/barry/projects/python/Lib/pickle.py", line 805, in load_global klass = self.find_class(module, name) File "/home/barry/projects/python/Lib/pickle.py", line 815, in find_class raise SystemError, \ SystemError: Failed to import class oops from module __builtin__ >>> import cPickle >>> cPickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? SystemError: Failed to import class oops from module __builtin__ ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 14:46 Message: Logged In: YES user_id=12800 Attaching a patch for pickle.py ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 From noreply@sourceforge.net Wed Nov 14 22:47:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 14:47:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-481882 ] pickle/cPickle can raise SystemError Message-ID: Bugs item #481882, was opened at 2001-11-14 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 6 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: pickle/cPickle can raise SystemError Initial Comment: pickle and cPickle can raise a SystemError when certain things fail during unpickling. For example, when attempting to unpickle a missing named reference (i.e. a class in a non-importable module, or a missing function in an existing module). See below. SystemError clearly is not the right exception to raise here, given the description of the error in the exceptions module (it tells you to contact your Python maintainer!). pickle and cPickle should probably simply propagate the original exception without masking it to SystemError. ----- Python 2.2b1+ (#3, Nov 13 2001, 18:06:11) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> pickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? File "/home/barry/projects/python/Lib/pickle.py", line 979, in loads return Unpickler(file).load() File "/home/barry/projects/python/Lib/pickle.py", line 588, in load dispatch[key](self) File "/home/barry/projects/python/Lib/pickle.py", line 805, in load_global klass = self.find_class(module, name) File "/home/barry/projects/python/Lib/pickle.py", line 815, in find_class raise SystemError, \ SystemError: Failed to import class oops from module __builtin__ >>> import cPickle >>> cPickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? SystemError: Failed to import class oops from module __builtin__ ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 14:47 Message: Logged In: YES user_id=12800 Attaching a patch for pickle.py ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 14:46 Message: Logged In: YES user_id=12800 Attaching a patch for pickle.py ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 From noreply@sourceforge.net Wed Nov 14 23:06:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 15:06:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-481882 ] pickle/cPickle can raise SystemError Message-ID: Bugs item #481882, was opened at 2001-11-14 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 6 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: pickle/cPickle can raise SystemError Initial Comment: pickle and cPickle can raise a SystemError when certain things fail during unpickling. For example, when attempting to unpickle a missing named reference (i.e. a class in a non-importable module, or a missing function in an existing module). See below. SystemError clearly is not the right exception to raise here, given the description of the error in the exceptions module (it tells you to contact your Python maintainer!). pickle and cPickle should probably simply propagate the original exception without masking it to SystemError. ----- Python 2.2b1+ (#3, Nov 13 2001, 18:06:11) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> pickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? File "/home/barry/projects/python/Lib/pickle.py", line 979, in loads return Unpickler(file).load() File "/home/barry/projects/python/Lib/pickle.py", line 588, in load dispatch[key](self) File "/home/barry/projects/python/Lib/pickle.py", line 805, in load_global klass = self.find_class(module, name) File "/home/barry/projects/python/Lib/pickle.py", line 815, in find_class raise SystemError, \ SystemError: Failed to import class oops from module __builtin__ >>> import cPickle >>> cPickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? SystemError: Failed to import class oops from module __builtin__ ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 15:06 Message: Logged In: YES user_id=12800 Attached a patch for cPickle.c ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 14:47 Message: Logged In: YES user_id=12800 Attaching a patch for pickle.py ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 14:46 Message: Logged In: YES user_id=12800 Attaching a patch for pickle.py ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 From noreply@sourceforge.net Wed Nov 14 23:13:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 15:13:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-481896 ] popen3 file objects hanging Message-ID: Bugs item #481896, was opened at 2001-11-14 15:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481896&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: popen3 file objects hanging Initial Comment: RE:#210676 System: Python 2.0 on solaris 8 The above bug is reported closed but I cannot see why. >>> sub = popen2.Popen3(cmd,1) >>> sub.fromchild.readlines() # blocked on stderr As reported in the above sited bug report, we too are suffering from this behaviour on file objects returned by popen3. It does appear that the stderr buffer is filling (the app writes a lot of stuff to stderr) and blocking stdout, but i cannot seem to flush the buffer. I need to acquire both stderr and stdout data. I have tried setting the bufsize to 0 (unbuffered) but that doesn't work. I have made the buffer big, but nothing there. Might there be some magic bufsize? popen4 (which stuffs both stderr and stdout into one file object) works perfectly as there is no blocking. Reading through the bugs about popen3, the bug report above is exactly what we are experiencing. The "solution" offered is not a solution at all. We are running 3rd party code, i.e. we can't close stderr inside the application, nor would we want to. stderr and stdout file descriptors must remain open for the duration of the application. Has there been some viable solution to this which has not been documented? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481896&group_id=5470 From noreply@sourceforge.net Thu Nov 15 05:17:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 21:17:17 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: Nobody/Anonymous (nobody) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Thu Nov 15 05:19:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 21:19:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: Nobody/Anonymous (nobody) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Thu Nov 15 05:39:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 21:39:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) >Assigned to: Tim Peters (tim_one) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Thu Nov 15 06:29:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 14 Nov 2001 22:29:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-481985 ] Uncaught MI order conflict Message-ID: Bugs item #481985, was opened at 2001-11-14 22:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481985&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: Uncaught MI order conflict Initial Comment: Dubious MRO from mixing classic and new-style classes in multiple inheritance: > def mrolist(k): > return ' '.join([c.__name__ for c in k.__mro__]) > > def f(): > # No question here. > class C: pass > class M1(C, object): pass > print mrolist(M1) # "M1 C object" > > # And no question here. > class D(C): pass > class M2(object, D): pass > print mrolist(M2) # "M2 object D C" > > # Here I expected > # "M3 M1 M2 object D C" > # but got > # "M3 M1 M2 D C object" > class M3(M1, M2): pass > print mrolist(M3) > > f() """ If I add class M4(M2, M1): pass print mrolist(M4) this prints what you may have expected: M4 M2 M1 object D C My conclusion is that it's an artefact of how conservative_merge() resolves the conflict between M1 and M2: in M1, C preceeds object, while in M2, object preceeds C. This order conflict is supposed to be outlawed, but I didn't feel like testing for it -- the function serious_order_disagreements() always returns false. In the light of conflicts, the algorithm in conservative() apparently favors the order of the first base class encountered. """ ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481985&group_id=5470 From noreply@sourceforge.net Thu Nov 15 08:26:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 00:26:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-482003 ] dict display broken in pprint Message-ID: Bugs item #482003, was opened at 2001-11-15 00:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482003&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Barry Warsaw (bwarsaw) Summary: dict display broken in pprint Initial Comment: pprint.PrettyPrinter.__format hoses display of multi- line dicts. Instead of placing the ":" after the key, it is placed before the key on continuation lines. The attached patch fixes this bug and adds a test case. Barry, once you're done with this you might want to reassign it to whoever is doing 2.1.2 in case this bug is present there as well. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482003&group_id=5470 From noreply@sourceforge.net Thu Nov 15 12:12:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 04:12:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: Tim Peters (tim_one) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Thu Nov 15 13:10:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 05:10:20 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) >Assigned to: Michael Hudson (mwh) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-11-15 05:10 Message: Logged In: YES user_id=6656 You seem to be editing PC/pyconfig.h, but I don't understand how this file is getting involved in a cygwin build. How are you building Python? It works just fine for me (except for curses, that's a cygwin bug). Also, have you got the latest version of cygwin? What version of windows, etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Thu Nov 15 14:30:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 06:30:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: Michael Hudson (mwh) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- >Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 06:30 Message: Logged In: YES user_id=52572 I have Cygwin 1.3.3-2. The latest appears to be 1.3.5-3, but I got mine just last week. They seem to release something new every few days. I'm using Windows 2000. I've also built and installed gcc-3.0.2 on my Cygwin installation. Oh, maybe this has something to do with it: I previously built a regular Win32 version of Python using MSVC from the same Python source tree. When I first tried to install Python for Cygwin, I made a separate build directory as they recommend for gcc, then did a "../Python-2.2b1/configure ; make" but it didn't work. If indeed the two builds are conflicting, you might consider making this approach possible. The idea is that no configuration-specific cruft gets left in the source tree. This has two advantages: 1. Rebuilds with the same source for different platforms becomes possible 2. You can build from a read-only copy of the sources. -Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 05:10 Message: Logged In: YES user_id=6656 You seem to be editing PC/pyconfig.h, but I don't understand how this file is getting involved in a cygwin build. How are you building Python? It works just fine for me (except for curses, that's a cygwin bug). Also, have you got the latest version of cygwin? What version of windows, etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Thu Nov 15 15:25:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 07:25:41 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) >Assigned to: Nobody/Anonymous (nobody) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-11-15 07:25 Message: Logged In: YES user_id=6656 What does "didn't work" mean? I suspect you've hit the nail on the head as to the cause of your problems, but I know diddly-squat about the VC++ build, so I'm unassigning this from me. I doubt it will be a priority for anyone else, sorry. Note you can do unix-style builds without polluting the source tree (much -- .pycs tend to get left lying around). ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 06:30 Message: Logged In: YES user_id=52572 I have Cygwin 1.3.3-2. The latest appears to be 1.3.5-3, but I got mine just last week. They seem to release something new every few days. I'm using Windows 2000. I've also built and installed gcc-3.0.2 on my Cygwin installation. Oh, maybe this has something to do with it: I previously built a regular Win32 version of Python using MSVC from the same Python source tree. When I first tried to install Python for Cygwin, I made a separate build directory as they recommend for gcc, then did a "../Python-2.2b1/configure ; make" but it didn't work. If indeed the two builds are conflicting, you might consider making this approach possible. The idea is that no configuration-specific cruft gets left in the source tree. This has two advantages: 1. Rebuilds with the same source for different platforms becomes possible 2. You can build from a read-only copy of the sources. -Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 05:10 Message: Logged In: YES user_id=6656 You seem to be editing PC/pyconfig.h, but I don't understand how this file is getting involved in a cygwin build. How are you building Python? It works just fine for me (except for curses, that's a cygwin bug). Also, have you got the latest version of cygwin? What version of windows, etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Thu Nov 15 16:16:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 08:16:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-479981 ] socket failed then compiled with treads Message-ID: Bugs item #479981, was opened at 2001-11-09 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 Category: Threads Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: socket failed then compiled with treads Initial Comment: Operating System HP-UX 11.0 (64 Bit) I tried to compile Python 2.1.1 this thread support. Doing so, the tests test_socket and test_asynchat both failed with "(9, 'Bad file number')" The test_socket succeds if I compiled python without threads, so I think it's an thread-problem. The thread-test istself succeeds. I think also, that python is correctly linked against libpthread. The same Problem occurs this Python 2.2.b1. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-15 08:15 Message: Logged In: NO 1. As expected, there is no change whether to lock gethostbyname or not. 2. gethostbyname is said to be threadsafe at least with HPUX 11.00 and should it be on further releases. HPUX 10.X don't support native Threads anyway 3. Python 2.1 run's on 32 Bit HP-UX-Machines 4. On 64 Bit HP-UX Machines I have found that some Patches are nessesary. I don't know what exact Patch is required but have some experiences with HP-UX General-Release Patch-Bundles: September 2000 - python failed June 2001 or later - python works maybe this helps someone ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-14 06:43 Message: Logged In: YES user_id=21627 Python puts a lock around gethostbyname if gethostbyname_r is not available. I don't think that should cause a problem even if gethostbyname is thread-safe. If you want to further analyse this, you could try to put an && !defined(__hpux) around the line #define USE_GETHOSTBYNAME_LOCK If you do, please report whether it works. In that case, it would be good to know whether gethostbyname is thread-safe on *all* HP-UX versions, or just on some. Furthermore, since this wasn't reported before: Is it specific to your installation, or reproducable on all installations? Is it perhaps specific to 64-bit mode? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 02:12 Message: Logged In: NO Maybe the failure has to do with gethostbyname-stuff. according the hpux-manpages theese reentrant interfaces are obsolete: int gethostent_r(struct hostent *result, struct hostent_data *buffer); int gethostbyname_r(const char *name, struct hostent *result, struct hostent_data *buffer); int gethostbyaddr_r(const char *addr, int len, int type, struct hostent *result, struct hostent_data *buffer); int sethostent_r(int stayopen, struct hostent_data *buffer); int endhostent_r(struct hostent_data *buffer); The configure-script has correctly found, that there is only a gethostbyname, but no gethostbyname_r. according to the man-pages the call to gethostbyname IS thread-save: struct hostent *gethostent(void); struct hostent *gethostbyname(const char *name); struct hostent *gethostbyaddr(const char *addr, int len, int type); ... MULTITHREAD USAGE Thread Safe: Yes Cancel Safe: Yes Async-cancel Safe: No Async-signal Safe: No Is it a failure to expect the gethostbyname to be not thread-save ? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 01:29 Message: Logged In: NO the complete compiler line used to link together the python binary: gcc -Wl,-E -Wl,+s -Wl,+b/mosquito/python/lib/python2.1/lib- dynload -o python \ Modules/python.o \ libpython2.1.a -lnsl -ldld -lpthread -lm The compiler I used is gcc 3.01. I tried it out with the cc from HP - the same effect After installing the Kernelpatch PHKL_25475, wich supersedes both PHKL_23842 and PHKL_20942 the failure remains the same ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 16:13 Message: Logged In: YES user_id=21627 Can you please also check whether the patches PHKL_23842 (or equivalent) and/or PHKL_20942. The patch titles are PHKL_23842 s700_800 11.00 signal,threads,spinlock,scheduler,IDS,q3p PHKL_20942 s700_800 11.00 signal, nanosleep, spinlock, scheduler It appears that HP-UX has a number of bugs where signals, forking, and other aspects of the operating functionality stop working in the presence of threads. If you don't have these patches, it would be good if you could try to install them, and report back whether it changes anything. Notice that HP apparently has a number of patches with these titles; they seem to be intended for different machines. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-09 15:51 Message: Logged In: YES user_id=21627 What compiler are you using? Can you please report the complete compiler line used to link together the python binary? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479981&group_id=5470 From noreply@sourceforge.net Thu Nov 15 16:29:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 08:29:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) >Assigned to: Michael Hudson (mwh) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- >Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 08:29 Message: Logged In: YES user_id=52572 "Didn't work" means that this #error in PyPort.h fired: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." How do you do unix-style builds without polluting the source tree? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 07:25 Message: Logged In: YES user_id=6656 What does "didn't work" mean? I suspect you've hit the nail on the head as to the cause of your problems, but I know diddly-squat about the VC++ build, so I'm unassigning this from me. I doubt it will be a priority for anyone else, sorry. Note you can do unix-style builds without polluting the source tree (much -- .pycs tend to get left lying around). ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 06:30 Message: Logged In: YES user_id=52572 I have Cygwin 1.3.3-2. The latest appears to be 1.3.5-3, but I got mine just last week. They seem to release something new every few days. I'm using Windows 2000. I've also built and installed gcc-3.0.2 on my Cygwin installation. Oh, maybe this has something to do with it: I previously built a regular Win32 version of Python using MSVC from the same Python source tree. When I first tried to install Python for Cygwin, I made a separate build directory as they recommend for gcc, then did a "../Python-2.2b1/configure ; make" but it didn't work. If indeed the two builds are conflicting, you might consider making this approach possible. The idea is that no configuration-specific cruft gets left in the source tree. This has two advantages: 1. Rebuilds with the same source for different platforms becomes possible 2. You can build from a read-only copy of the sources. -Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 05:10 Message: Logged In: YES user_id=6656 You seem to be editing PC/pyconfig.h, but I don't understand how this file is getting involved in a cygwin build. How are you building Python? It works just fine for me (except for curses, that's a cygwin bug). Also, have you got the latest version of cygwin? What version of windows, etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Thu Nov 15 16:50:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 08:50:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) >Assigned to: Nobody/Anonymous (nobody) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-11-15 08:50 Message: Logged In: YES user_id=6656 Right, OK. By the time-honoured mkdir build && cd build && ../configure && make trick. By unix-like, I mean unix or cygwin. It seems that doing a VC++ build does something to the tree that prevents such builds working, but as I don't have VC++, I can't say what or what to do about it -- so why did you reassign the bug to me? ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 08:29 Message: Logged In: YES user_id=52572 "Didn't work" means that this #error in PyPort.h fired: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." How do you do unix-style builds without polluting the source tree? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 07:25 Message: Logged In: YES user_id=6656 What does "didn't work" mean? I suspect you've hit the nail on the head as to the cause of your problems, but I know diddly-squat about the VC++ build, so I'm unassigning this from me. I doubt it will be a priority for anyone else, sorry. Note you can do unix-style builds without polluting the source tree (much -- .pycs tend to get left lying around). ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 06:30 Message: Logged In: YES user_id=52572 I have Cygwin 1.3.3-2. The latest appears to be 1.3.5-3, but I got mine just last week. They seem to release something new every few days. I'm using Windows 2000. I've also built and installed gcc-3.0.2 on my Cygwin installation. Oh, maybe this has something to do with it: I previously built a regular Win32 version of Python using MSVC from the same Python source tree. When I first tried to install Python for Cygwin, I made a separate build directory as they recommend for gcc, then did a "../Python-2.2b1/configure ; make" but it didn't work. If indeed the two builds are conflicting, you might consider making this approach possible. The idea is that no configuration-specific cruft gets left in the source tree. This has two advantages: 1. Rebuilds with the same source for different platforms becomes possible 2. You can build from a read-only copy of the sources. -Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 05:10 Message: Logged In: YES user_id=6656 You seem to be editing PC/pyconfig.h, but I don't understand how this file is getting involved in a cygwin build. How are you building Python? It works just fine for me (except for curses, that's a cygwin bug). Also, have you got the latest version of cygwin? What version of windows, etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Thu Nov 15 16:56:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 08:56:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: Nobody/Anonymous (nobody) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- >Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 08:56 Message: Logged In: YES user_id=52572 Sorry, I just assumed you wouldn't be notified that I still had a question if the bug stayed unassigned. I don't know much about how this tracker is set up. I guess when I tried the technique you propose, the directory was already polluted from the Win32 build. Thanks for all your help, Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 08:50 Message: Logged In: YES user_id=6656 Right, OK. By the time-honoured mkdir build && cd build && ../configure && make trick. By unix-like, I mean unix or cygwin. It seems that doing a VC++ build does something to the tree that prevents such builds working, but as I don't have VC++, I can't say what or what to do about it -- so why did you reassign the bug to me? ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 08:29 Message: Logged In: YES user_id=52572 "Didn't work" means that this #error in PyPort.h fired: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." How do you do unix-style builds without polluting the source tree? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 07:25 Message: Logged In: YES user_id=6656 What does "didn't work" mean? I suspect you've hit the nail on the head as to the cause of your problems, but I know diddly-squat about the VC++ build, so I'm unassigning this from me. I doubt it will be a priority for anyone else, sorry. Note you can do unix-style builds without polluting the source tree (much -- .pycs tend to get left lying around). ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 06:30 Message: Logged In: YES user_id=52572 I have Cygwin 1.3.3-2. The latest appears to be 1.3.5-3, but I got mine just last week. They seem to release something new every few days. I'm using Windows 2000. I've also built and installed gcc-3.0.2 on my Cygwin installation. Oh, maybe this has something to do with it: I previously built a regular Win32 version of Python using MSVC from the same Python source tree. When I first tried to install Python for Cygwin, I made a separate build directory as they recommend for gcc, then did a "../Python-2.2b1/configure ; make" but it didn't work. If indeed the two builds are conflicting, you might consider making this approach possible. The idea is that no configuration-specific cruft gets left in the source tree. This has two advantages: 1. Rebuilds with the same source for different platforms becomes possible 2. You can build from a read-only copy of the sources. -Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 05:10 Message: Logged In: YES user_id=6656 You seem to be editing PC/pyconfig.h, but I don't understand how this file is getting involved in a cygwin build. How are you building Python? It works just fine for me (except for curses, that's a cygwin bug). Also, have you got the latest version of cygwin? What version of windows, etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Thu Nov 15 17:06:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 09:06:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-482171 ] webchecker dies on file: URLs w/o robots Message-ID: Bugs item #482171, was opened at 2001-11-15 09:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482171&group_id=5470 Category: Demos and Tools Group: None Status: Open Resolution: None Priority: 5 Submitted By: Fred L. Drake, Jr. (fdrake) Assigned to: Nobody/Anonymous (nobody) Summary: webchecker dies on file: URLs w/o robots Initial Comment: When webchecker is run on local documents using a file: URL, it dies if there is not a robots.txt in the root directory: cj42289-a(.../python/Doc); python ../Tools/webchecker/webchecker.py file:`pwd`/html/api/ webchecker version 1.24 Traceback (most recent call last): File "../Tools/webchecker/webchecker.py", line 858, in ? main() File "../Tools/webchecker/webchecker.py", line 205, in main c.addroot(arg) File "../Tools/webchecker/webchecker.py", line 324, in addroot self.addrobot(root) File "../Tools/webchecker/webchecker.py", line 337, in addrobot rp.read() File "/usr/local/lib/python2.2/robotparser.py", line 43, in read f = opener.open(self.url) File "/usr/local/lib/python2.2/urllib.py", line 178, in open return getattr(self, name)(url) File "/usr/local/lib/python2.2/urllib.py", line 405, in open_file return self.open_local_file(url) File "/usr/local/lib/python2.2/urllib.py", line 412, in open_local_file stats = os.stat(localname) OSError: [Errno 2] No such file or directory: '/robots.txt' ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482171&group_id=5470 From noreply@sourceforge.net Thu Nov 15 17:34:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 09:34:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-476419 ] Turtle documentation not included. Message-ID: Bugs item #476419, was opened at 2001-10-30 09:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476419&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 3 Submitted By: Josh Cogliati (jjc) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Turtle documentation not included. Initial Comment: Problem: The file libturtle.tex has been written. However it is not included in lib.tex Also in libundoc.tex the turtle library is listed as undocumented, dispite the fact that documentation has been written. Solution: Include libturtle.tex in lib.tex and remove the references to turtle being undocumented in libundoc.tex ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-15 09:34 Message: Logged In: YES user_id=3066 The turtle docs have now been added (Doc/lib/lib.tex revision 1.196) as part of Python 2.2 beta 2. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-01 12:37 Message: Logged In: YES user_id=3066 While this is seemingly easy to fix, the real problem (which is why the turtle docs have never been included), is there there hasn't been a good place to put it (no GUI or Tkinter chapter). Fortunately, this is being corrected, so we should see this section included with Python 2.2 (probably in beta 2). Dropping the priority since it will be taken care of as part of an existing effort. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476419&group_id=5470 From noreply@sourceforge.net Thu Nov 15 17:35:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 09:35:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-464900 ] Include TKInter Docs Message-ID: Bugs item #464900, was opened at 2001-09-25 11:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=464900&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 4 Submitted By: paul rubin (phr) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Include TKInter Docs Initial Comment: The standard Python distributions include the TKinter library but no docs for it. Arggh! I hope this can be remedied. The docs I've seen aren't so great, but they're enough to get started. Please consider tossing them into the Python distribution even if there's nobody available to edit them further. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-15 09:35 Message: Logged In: YES user_id=3066 Preliminary Tkinter docs have been checked in and will be part of Python 2.2 beta 2. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-10-24 19:11 Message: Logged In: YES user_id=3066 I'm not ignoring this, and one person is working on some docs for the Tk-related modules, but the existing published docs are not available to be tossed-in -- someone else wrote them and owns the copyright. Rest assured, this will be remedied, but it won't be quick given the number of documentation issues that need to be handled at this time. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=464900&group_id=5470 From noreply@sourceforge.net Thu Nov 15 23:15:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 15:15:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: Tim Peters (tim_one) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-15 15:15 Message: Logged In: YES user_id=31435 The source distro is just a snapshot of Python's CVS development tree, so it's a build tree, not an installation tree. "The std" Windows installation tree is the one created by the PythonLabs Windows installer. Similar Windows trees are created by the ActiveState and PythonWare Windows installers. "The std" Unix installation tree is whatever the heck "make install" does on Unix (and it's not the *build* tree either). Likewise for Macs, which rearrage the build tree in yet other ways for installation. The relationship between the build tree and the Windows installation tree is established by the 3000-line python20.wse script in PCbuild, which is input to the Wise installer-builder; I can't summarize it usefully here, and Windows installation also involves fiddling with registry entries. Afraid I'm still not clear on why you're asking, although I'm quite sure I'm not really helping . ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Thu Nov 15 23:19:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 15:19:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-210637 ] ihooks on windows and pythoncom (PR#294) Message-ID: Bugs item #210637, was opened at 2000-07-31 14:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210637&group_id=5470 Category: Windows Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Mark Hammond (mhammond) Summary: ihooks on windows and pythoncom (PR#294) Initial Comment: Jitterbug-Id: 294 Submitted-By: mak@mikroplan.com.pl Date: Thu, 13 Apr 2000 04:09:35 -0400 (EDT) Version: cvs OS: windows Hi, Python module ihooks is not so compatible with builtin imp while importing modules whose name is stored in registry eg. pythoncom/pywintypes. import ihooks ihooks.install() import pythoncom This code will fail inside pythonwin ide too ! ==================================================================== Audit trail: Tue Jul 11 08:29:17 2000 guido moved from incoming to open ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-15 15:19 Message: Logged In: NO ¾È³çÇϼ¼¿ä ÀÌÁ¦ºÎÅÍ Á¦ ¼Ò°³¸¦ ÇÏ°Ú½À´Ï´Ù ÀúÀÇ À̸§Àº ¹ÚÇýÁØ ÀÌ°í¿ä ³ªÀÌ´Â 13»ìÀÌ¿¡¿ä ±×¸®°í °¡Á·Àº ¸ðµÎ 4¸í ¾ö¸¶ ¾Æºü ´©³ª ³ª Á¦°¡ »ç´Â °÷Àº ºÐ´ç±¸ ¾ßžµ¿ ¸ÅÈ­¸¶À» 105-1006 Á¦ ÀüÈ­¹øÈ£´Â¿© 031-704-9838 Àú´Â Çѱ¹ÀÎ ÀÔ´Ï´Ù. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-09-19 10:23 Message: Logged In: NO ruoy retupmoc si daed ---------------------------------------------------------------------- Comment By: Grzegorz Makarewicz (mpmak) Date: 2001-03-02 04:27 Message: Logged In: YES user_id=141704 BasicModuleLoader.find_module_in_dir is searching for main modules only in frozen and builtin. The imp searches the registry, too. ModuleLoader.find_module_in_dir should call the functions from the inherited object. so this patch should help: --- V:\py21\Lib\ihooks.py Mon Feb 12 08:55:46 2001 +++ ihooks.py Sun Feb 18 04:39:39 2001 @@ -122,8 +122,13 @@ def find_module_in_dir(self, name, dir): if dir is None: - return self.find_builtin_module(name) - else: + result = self.find_builtin_module(name) + if result is not None: + return result + try: + return imp.find_module(name, None) + except: + return None try: return imp.find_module(name, [dir]) except ImportError: @@ -237,7 +242,7 @@ def find_module_in_dir(self, name, dir, allow_packages=1): if dir is None: - return self.find_builtin_module(name) + return BasicModuleLoader.find_module_in_dir (self,name,dir) if allow_packages: fullname = self.hooks.path_join(dir, name) if self.hooks.path_isdir(fullname): ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2000-08-30 23:23 Message: Leaving open, but moving down the priority and resolution lists. A patch would help bump it back up :-) ---------------------------------------------------------------------- Comment By: Mark Hammond (mhammond) Date: 2000-08-13 23:42 Message: This needs a resolution. The "registered module" code in the code also needs to support HKEY_CURRENT_USER along with the HKEY_LOCAL_MACHINE it does now. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210637&group_id=5470 From noreply@sourceforge.net Thu Nov 15 23:34:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 15:34:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: Tim Peters (tim_one) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 15:34 Message: Logged In: YES user_id=52572 The problem goes something like this: Assume I have a nice, generalized build system (I do, but not as nice as it will be soon ;-)), to which I want to add Python support. I want it to be possible to compile/test code against multiple Python versions, though that's not too relevant to the problem. I'd like finding the neccessary components (headers, libs, executables) to be relatively automatic, based on, e.g. the user's specification of a python base directory and a version number. On Unices it should be possible to do without the base directory unless python was configured --with-prefix=.... I have to assume that someone developing Python extension modules may either be using the source tree, or an installation specific to to the platform he's developing on. Someone who's just building an extension module is probably using a Python installation, but I can't be certain. So, does that clear things up a bit? -Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-15 15:15 Message: Logged In: YES user_id=31435 The source distro is just a snapshot of Python's CVS development tree, so it's a build tree, not an installation tree. "The std" Windows installation tree is the one created by the PythonLabs Windows installer. Similar Windows trees are created by the ActiveState and PythonWare Windows installers. "The std" Unix installation tree is whatever the heck "make install" does on Unix (and it's not the *build* tree either). Likewise for Macs, which rearrage the build tree in yet other ways for installation. The relationship between the build tree and the Windows installation tree is established by the 3000-line python20.wse script in PCbuild, which is input to the Wise installer-builder; I can't summarize it usefully here, and Windows installation also involves fiddling with registry entries. Afraid I'm still not clear on why you're asking, although I'm quite sure I'm not really helping . ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Thu Nov 15 23:49:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 15:49:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-481882 ] pickle/cPickle can raise SystemError Message-ID: Bugs item #481882, was opened at 2001-11-14 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 6 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: pickle/cPickle can raise SystemError Initial Comment: pickle and cPickle can raise a SystemError when certain things fail during unpickling. For example, when attempting to unpickle a missing named reference (i.e. a class in a non-importable module, or a missing function in an existing module). See below. SystemError clearly is not the right exception to raise here, given the description of the error in the exceptions module (it tells you to contact your Python maintainer!). pickle and cPickle should probably simply propagate the original exception without masking it to SystemError. ----- Python 2.2b1+ (#3, Nov 13 2001, 18:06:11) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import pickle >>> pickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? File "/home/barry/projects/python/Lib/pickle.py", line 979, in loads return Unpickler(file).load() File "/home/barry/projects/python/Lib/pickle.py", line 588, in load dispatch[key](self) File "/home/barry/projects/python/Lib/pickle.py", line 805, in load_global klass = self.find_class(module, name) File "/home/barry/projects/python/Lib/pickle.py", line 815, in find_class raise SystemError, \ SystemError: Failed to import class oops from module __builtin__ >>> import cPickle >>> cPickle.loads('c__builtin__\noops\np0\n.') Traceback (most recent call last): File "", line 1, in ? SystemError: Failed to import class oops from module __builtin__ ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-15 15:49 Message: Logged In: YES user_id=12800 Fixed in Python 2.2beta2, cPickle.c 2.70, pickle.py 1.55 ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 15:06 Message: Logged In: YES user_id=12800 Attached a patch for cPickle.c ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 14:47 Message: Logged In: YES user_id=12800 Attaching a patch for pickle.py ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-14 14:46 Message: Logged In: YES user_id=12800 Attaching a patch for pickle.py ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481882&group_id=5470 From noreply@sourceforge.net Fri Nov 16 00:12:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 16:12:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-15 16:12 Message: Logged In: YES user_id=12800 I have rewritten the pickle documentation, and updated the marshal, cPickle, and copy_reg documentation. I believe all the documentation issues raised here have been fixed. Implementation issues will be pushed to Python 2.3, and the plan is to write a PEP to plan future work. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:39 Message: Logged In: YES user_id=12800 Assigning to Jeremy so he'll remember to forward me Jim's email. Jeremy can ping-pong it to Fred if he wants, and then reassign to me after forwarding the email. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Fri Nov 16 00:43:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 16:43:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Closed Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-15 16:43 Message: Logged In: NO Barry, can you also do something about the Cookie module, or at least about its documentation? If Cookie is aliased to SmartCookie, I think that needs mention. --Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-15 16:12 Message: Logged In: YES user_id=12800 I have rewritten the pickle documentation, and updated the marshal, cPickle, and copy_reg documentation. I believe all the documentation issues raised here have been fixed. Implementation issues will be pushed to Python 2.3, and the plan is to write a PEP to plan future work. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:39 Message: Logged In: YES user_id=12800 Assigning to Jeremy so he'll remember to forward me Jim's email. Jeremy can ping-pong it to Fred if he wants, and then reassign to me after forwarding the email. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Fri Nov 16 03:23:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 15 Nov 2001 19:23:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-478038 ] urlparse.urlparse semicolon bug Message-ID: Bugs item #478038, was opened at 2001-11-04 09:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 Category: Python Library Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urlparse.urlparse semicolon bug Initial Comment: urlparse,urlparse uses obsolete parsing rules. It expects there to be no more than one semicolon in a URL, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit?x=y It splits the url into parts, one of which is the part after between the semicolon and the question mark. This behavior is based on an obsolete URL spec. Recent specs, including the RFCs referenced in the urlparse documentation allow semicolons in each path, as in: http://127.0.0.1:8880/semitest/foo;presentation=edit/form/spam;eggs=1/splat urlparse.urlparse parses as follows: [jim@c ZServer]$ python2.2 Python 2.2b1 (#1, Oct 22 2001, 17:42:33) [GCC 2.95.3 19991030 (prerelease)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Py$ from urlparse import urlparse Py$ urlparse("http://127.0.0.1:8880/semitest/foo%3Bbar;presentation=edit/form/spam;eggs=1/splat") ('http', '127.0.0.1:8880', '/semitest/foo%3Bbar', 'presentation=edit/form/spam;eggs=1/splat', '', '') Py$ which is incorrect because much of the path is incorrectly included in the obsolete "params" part. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-15 19:23 Message: Logged In: YES user_id=3066 Fixed in Lib/urlparse.py 1.31 and 1.30.10.1. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 14:47 Message: Logged In: YES user_id=3066 I've attached a patch that makes the change described in my previous comment and cleans up the code a little. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-05 13:35 Message: Logged In: YES user_id=3066 Here's my proposal for a fix: For the existing urlparse() function, return something in the params field of the result tuple only if it appears on the last path segment. This makes it an empty string for your example, but for URLs which conform to the simpler version of the specifications the API was designed for continue to give the expected behavior. To support the current RFC 2396 syntax, a new function is needed which returns a 5-tuple (the current 6-tuple less the params field). A second new function can be provided which splits the path component into a sequence of pairs, which each pair is (namepart, params). Does this seem acceptable? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478038&group_id=5470 From noreply@sourceforge.net Fri Nov 16 13:22:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 05:22:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-482460 ] dumbdbm fails to overwrite existing key Message-ID: Bugs item #482460, was opened at 2001-11-16 05:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482460&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael McCandless (mikemccand) Assigned to: Nobody/Anonymous (nobody) Summary: dumbdbm fails to overwrite existing key Initial Comment: Here's a simple test case that shows the bug: import dumbdbm db = dumbdbm.open('db', 'c') db['1'] = 'hello' db['1'] = 'hello2' db.close() db = dumbdbm.open('db', 'c') print db['1'] db.close() This prints out 'hello' but should print out 'hello2'. As far as I can tell, this bug did not exist 1.5.2, but then appeared in 2.0 and is still present in 2.2b1. The problem is, when overwriting the key, dumbdbm fails to update the .dir file. I believe this patch fixes the bug: > diff -c Lib/dumbdbm.py Lib/dumbdbm.py.new *** Lib/dumbdbm.py Tue Sep 4 15:14:13 2001 --- Lib/dumbdbm.py.new Fri Nov 16 08:16:42 2001 *************** *** 124,129 **** --- 124,132 ---- else: pos, siz = self._addval(val) self._index[key] = pos, siz + tup = (pos, siz) + if self._index[key] != tup: + self._addkey(key, tup) def __delitem__(self, key): del self._index[key] ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482460&group_id=5470 From noreply@sourceforge.net Fri Nov 16 13:38:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 05:38:17 -0800 Subject: [Python-bugs-list] [ python-Bugs-482460 ] dumbdbm fails to overwrite existing key Message-ID: Bugs item #482460, was opened at 2001-11-16 05:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482460&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael McCandless (mikemccand) Assigned to: Nobody/Anonymous (nobody) Summary: dumbdbm fails to overwrite existing key Initial Comment: Here's a simple test case that shows the bug: import dumbdbm db = dumbdbm.open('db', 'c') db['1'] = 'hello' db['1'] = 'hello2' db.close() db = dumbdbm.open('db', 'c') print db['1'] db.close() This prints out 'hello' but should print out 'hello2'. As far as I can tell, this bug did not exist 1.5.2, but then appeared in 2.0 and is still present in 2.2b1. The problem is, when overwriting the key, dumbdbm fails to update the .dir file. I believe this patch fixes the bug: > diff -c Lib/dumbdbm.py Lib/dumbdbm.py.new *** Lib/dumbdbm.py Tue Sep 4 15:14:13 2001 --- Lib/dumbdbm.py.new Fri Nov 16 08:16:42 2001 *************** *** 124,129 **** --- 124,132 ---- else: pos, siz = self._addval(val) self._index[key] = pos, siz + tup = (pos, siz) + if self._index[key] != tup: + self._addkey(key, tup) def __delitem__(self, key): del self._index[key] ---------------------------------------------------------------------- >Comment By: Michael McCandless (mikemccand) Date: 2001-11-16 05:38 Message: Logged In: YES user_id=323786 Sorry -- that previous patch does not work. I believe this one does: > diff -c Lib/dumbdbm.py Lib/dumbdbm.py.new *** Lib/dumbdbm.py Fri Nov 16 08:36:36 2001 --- Lib/dumbdbm.py.new Fri Nov 16 08:36:50 2001 *************** *** 116,121 **** --- 116,122 ---- self._addkey(key, (pos, siz)) else: pos, siz = self._index[key] + startTup = (pos, siz) oldblocks = (siz + _BLOCKSIZE - 1) / _BLOCKSIZE newblocks = (len(val) + _BLOCKSIZE - 1) / _BLOCKSIZE if newblocks <= oldblocks: *************** *** 124,129 **** --- 125,133 ---- else: pos, siz = self._addval(val) self._index[key] = pos, siz + tup = (pos, siz) + if startTup != tup: + self._addkey(key, tup) def __delitem__(self, key): del self._index[key] ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482460&group_id=5470 From noreply@sourceforge.net Fri Nov 16 15:26:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 07:26:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-410620 ] pydoc installation w.r.t. RPM. Message-ID: Bugs item #410620, was opened at 2001-03-22 15:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=410620&group_id=5470 Category: Installation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Ka-Ping Yee (ping) Summary: pydoc installation w.r.t. RPM. Initial Comment: Overview: When building an RPM the value for the --prefix configure variable takes on double meaning. Why? Well --prefix tells the build system to "build python so that if runs from --prefix at run time". This infers that during "make install" you want to install python in the --prefix area. The problem is when building an RPM you need to build python for --prefix but install into a temprory place (not indicated by the --prefix variable). This is done by overriding the environment variable $prefix during "make install" so rpm can install into a temprary location in order to snarf up the files for packaging. Python in general plays this game well however from 2.1a2 to 2.1b1 the pydoc stuff breaks this paradigm. I'm using this patch in my 2.1b1 rpm specfile but I would prefer the patch to be incorporated into the base installation. -res Patch included below: --------------------- snip ------------------- *** Python-2.1b1/Makefile.pre.in.ORIG Wed Mar 21 13:24:09 2001 --- Python-2.1b1/Makefile.pre.in Wed Mar 21 13:24:48 2001 *************** *** 694,700 **** # This goes into $(exec_prefix) sharedinstall: PYTHONPATH= ./$(PYTHON) $(srcdir)/setup.py install \ ! --install-platlib=$(DESTSHARED) # Build the toplevel Makefile Makefile.pre: Makefile.pre.in config.status --- 694,700 ---- # This goes into $(exec_prefix) sharedinstall: PYTHONPATH= ./$(PYTHON) $(srcdir)/setup.py install \ ! --install-platlib=$(DESTSHARED) --prefix=${prefix} # Build the toplevel Makefile Makefile.pre: Makefile.pre.in config.status ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-16 07:25 Message: Logged In: NO I run into the same problem when trying to build/install pyhton-2.1.1 without write-access to /usr/local/bin/ using prefix=... it took quite some time util I understood the problem (it's my first try with python) and found the same fix. why isn't this _correct_ patch included ??? ---------------------------------------------------------------------- Comment By: Anthon van der Neut (anthon) Date: 2001-04-30 04:14 Message: Logged In: YES user_id=90778 I ran into the same problem with 2.1 final release. Tried to use the --root option in that place only to find that the .so files installed by sharedinstall: where now in $(RPM_BUILD_ROOT)/$(RPM_BUILD_ROOT)/.... :( I chose the more restricted patch: --- Makefile.pre.in.org +++ Makefile.pre.in Mon Apr 30 11:33:53 2001 @@ -704,7 +708,7 @@ # This goes into $(exec_prefix) sharedinstall: PYTHONPATH= ./$(PYTHON) $(srcdir)/setup.py install \ - --install-platlib=$(DESTSHARED) + --install-platlib=$(DESTSHARED) --install- scripts=$(BINDIR) # Build the toplevel Makefile Makefile.pre: Makefile.pre.in config.status ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=410620&group_id=5470 From noreply@sourceforge.net Fri Nov 16 16:04:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 08:04:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-482510 ] bug in slicing time.struct_time Message-ID: Bugs item #482510, was opened at 2001-11-16 08:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482510&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bug in slicing time.struct_time Initial Comment: There seems to be a bug in the slicing of time.struct_time Getting a single item works but a slice doesn't if you use a non-zero start. i.e. t[i] works and t[:j] seems to be ok, but t[i:] and t[i:j] don't work - they seem to overwrite the first i items with see the transcript below for examples: >>> import time >>> time.localtime() (2001, 11, 16, 21, 17, 9, 4, 320, 0) >>> time.localtime()[3:6] (, , ) >>> time.localtime()[-2] 320 >>> time.localtime()[-2:] (, ) >>> time.localtime()[:-2] (2001, 11, 16, 21, 17, 56, 4) >>> t = time.struct_time((2001, 11, 16, 21, 18, 0, 4, 320, 0)) >>> t.tm_year 2001 >>> t[3:6] (, , ) >>> t[0:] (2001, 11, 16, 21, 18, 0, 4, 320, 0) >>> t[:0] () >>> t[:-1] (2001, 11, 16, 21, 18, 0, 4, 320) >>> t[:-3] (2001, 11, 16, 21, 18, 0) >>> t[1:] (, 11, 16, 21, 18, 0, 4, 320) >>> t[4:] (, , , , 18) >>> t.__getslice__(2, 5) (, , 16) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482510&group_id=5470 From noreply@sourceforge.net Fri Nov 16 16:16:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 08:16:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-482460 ] dumbdbm fails to overwrite existing key Message-ID: Bugs item #482460, was opened at 2001-11-16 05:22 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482460&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael McCandless (mikemccand) Assigned to: Nobody/Anonymous (nobody) Summary: dumbdbm fails to overwrite existing key Initial Comment: Here's a simple test case that shows the bug: import dumbdbm db = dumbdbm.open('db', 'c') db['1'] = 'hello' db['1'] = 'hello2' db.close() db = dumbdbm.open('db', 'c') print db['1'] db.close() This prints out 'hello' but should print out 'hello2'. As far as I can tell, this bug did not exist 1.5.2, but then appeared in 2.0 and is still present in 2.2b1. The problem is, when overwriting the key, dumbdbm fails to update the .dir file. I believe this patch fixes the bug: > diff -c Lib/dumbdbm.py Lib/dumbdbm.py.new *** Lib/dumbdbm.py Tue Sep 4 15:14:13 2001 --- Lib/dumbdbm.py.new Fri Nov 16 08:16:42 2001 *************** *** 124,129 **** --- 124,132 ---- else: pos, siz = self._addval(val) self._index[key] = pos, siz + tup = (pos, siz) + if self._index[key] != tup: + self._addkey(key, tup) def __delitem__(self, key): del self._index[key] ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-11-16 08:16 Message: Logged In: YES user_id=44345 Performance isn't a big issue with dumbdbm, so I think the code should be as simple as possible. I have a local version I was working on a bit. I simply delete then add in __setitem__: def __setitem__(self, key, val): if not type(key) == type('') == type(val): raise TypeError, "keys and values must be strings" if self._index.has_key(key): del self[key] (pos, siz) = self._addval(val) self._addkey(key, (pos, siz)) I also have code in my local version to reuse space though. I also want to add file locking (which is the motivation for me to be fiddling with this stuff). ---------------------------------------------------------------------- Comment By: Michael McCandless (mikemccand) Date: 2001-11-16 05:38 Message: Logged In: YES user_id=323786 Sorry -- that previous patch does not work. I believe this one does: > diff -c Lib/dumbdbm.py Lib/dumbdbm.py.new *** Lib/dumbdbm.py Fri Nov 16 08:36:36 2001 --- Lib/dumbdbm.py.new Fri Nov 16 08:36:50 2001 *************** *** 116,121 **** --- 116,122 ---- self._addkey(key, (pos, siz)) else: pos, siz = self._index[key] + startTup = (pos, siz) oldblocks = (siz + _BLOCKSIZE - 1) / _BLOCKSIZE newblocks = (len(val) + _BLOCKSIZE - 1) / _BLOCKSIZE if newblocks <= oldblocks: *************** *** 124,129 **** --- 125,133 ---- else: pos, siz = self._addval(val) self._index[key] = pos, siz + tup = (pos, siz) + if startTup != tup: + self._addkey(key, tup) def __delitem__(self, key): del self._index[key] ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482460&group_id=5470 From noreply@sourceforge.net Fri Nov 16 16:26:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 08:26:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-482510 ] bug in slicing time.struct_time Message-ID: Bugs item #482510, was opened at 2001-11-16 08:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482510&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bug in slicing time.struct_time Initial Comment: There seems to be a bug in the slicing of time.struct_time Getting a single item works but a slice doesn't if you use a non-zero start. i.e. t[i] works and t[:j] seems to be ok, but t[i:] and t[i:j] don't work - they seem to overwrite the first i items with see the transcript below for examples: >>> import time >>> time.localtime() (2001, 11, 16, 21, 17, 9, 4, 320, 0) >>> time.localtime()[3:6] (, , ) >>> time.localtime()[-2] 320 >>> time.localtime()[-2:] (, ) >>> time.localtime()[:-2] (2001, 11, 16, 21, 17, 56, 4) >>> t = time.struct_time((2001, 11, 16, 21, 18, 0, 4, 320, 0)) >>> t.tm_year 2001 >>> t[3:6] (, , ) >>> t[0:] (2001, 11, 16, 21, 18, 0, 4, 320, 0) >>> t[:0] () >>> t[:-1] (2001, 11, 16, 21, 18, 0, 4, 320) >>> t[:-3] (2001, 11, 16, 21, 18, 0) >>> t[1:] (, 11, 16, 21, 18, 0, 4, 320) >>> t[4:] (, , , , 18) >>> t.__getslice__(2, 5) (, , 16) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-16 08:26 Message: Logged In: NO Addendum to the above: Sorry, I forgot to mention the platform - this is with the binary distribution of 2.2b1 for Windows (i.e. Python-2.2b1.exe). I found the problem on Win98 & Win2K - haven't tried any other platforms yet. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482510&group_id=5470 From noreply@sourceforge.net Fri Nov 16 16:41:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 08:41:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-482510 ] bug in slicing time.struct_time Message-ID: Bugs item #482510, was opened at 2001-11-16 08:04 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482510&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Michael Hudson (mwh) Summary: bug in slicing time.struct_time Initial Comment: There seems to be a bug in the slicing of time.struct_time Getting a single item works but a slice doesn't if you use a non-zero start. i.e. t[i] works and t[:j] seems to be ok, but t[i:] and t[i:j] don't work - they seem to overwrite the first i items with see the transcript below for examples: >>> import time >>> time.localtime() (2001, 11, 16, 21, 17, 9, 4, 320, 0) >>> time.localtime()[3:6] (, , ) >>> time.localtime()[-2] 320 >>> time.localtime()[-2:] (, ) >>> time.localtime()[:-2] (2001, 11, 16, 21, 17, 56, 4) >>> t = time.struct_time((2001, 11, 16, 21, 18, 0, 4, 320, 0)) >>> t.tm_year 2001 >>> t[3:6] (, , ) >>> t[0:] (2001, 11, 16, 21, 18, 0, 4, 320, 0) >>> t[:0] () >>> t[:-1] (2001, 11, 16, 21, 18, 0, 4, 320) >>> t[:-3] (2001, 11, 16, 21, 18, 0) >>> t[1:] (, 11, 16, 21, 18, 0, 4, 320) >>> t[4:] (, , , , 18) >>> t.__getslice__(2, 5) (, , 16) ---------------------------------------------------------------------- >Comment By: Michael Hudson (mwh) Date: 2001-11-16 08:41 Message: Logged In: YES user_id=6656 This has been fixed in CVS. See http://mail.python.org/pipermail/python-dev/2001-October/018272.html for more info. Jack, this is why you should use the bug tracker . ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-16 08:26 Message: Logged In: NO Addendum to the above: Sorry, I forgot to mention the platform - this is with the binary distribution of 2.2b1 for Windows (i.e. Python-2.2b1.exe). I found the problem on Win98 & Win2K - haven't tried any other platforms yet. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482510&group_id=5470 From noreply@sourceforge.net Fri Nov 16 16:54:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 08:54:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-482531 ] python directory not added to PATH Message-ID: Bugs item #482531, was opened at 2001-11-16 08:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482531&group_id=5470 Category: Windows Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Alexandre Fayolle (afayolle) Assigned to: Tim Peters (tim_one) Summary: python directory not added to PATH Initial Comment: On installation on Windows platforms, the python directory is not added to the PATH, which makes it difficult to launch python through batch files for instance. Alexandre Fayolle ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482531&group_id=5470 From noreply@sourceforge.net Fri Nov 16 18:56:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 10:56:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-482574 ] audioop.ratecv crashes Message-ID: Bugs item #482574, was opened at 2001-11-16 10:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482574&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Matthias Baas (mbaas) Assigned to: Nobody/Anonymous (nobody) Summary: audioop.ratecv crashes Initial Comment: The ratecv in the audioop module can crash under certain conditions. I'm using Python 2.1.1 under Win98 SE and SuSE Linux 7.2 Here's a small program that demonstrates the problem. It creates an empty sample and tries to call ratecv. ################################################## import audioop nchannels = 2 width = 2 framerate = 44100 nframes = 107880 frames = nchannels*width*nframes*chr(0) newrate = 37083 #newrate = 35002 newfrag,state = audioop.ratecv(frames, width, nchannels, framerate, newrate, None) ################################################## With newrate = 37083 it crashes and with 35002 it creates a MemoryError exception. I already had a look into the source of audioop and found the problem to be in the following line inside audioop_ratecv(): str = PyString_FromStringAndSize( NULL, size * nchannels * (len * outrate + inrate - 1) / inrate); With big enough samples the second argument simply overflows. In the case of newrate=35002 the above expression yields a negative value and so an MemoryError exception occurs. With newrate=37083 the expression yields a positive value, but one that's much too small, so the remainder of the function writes into unallocated memory and crashes. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482574&group_id=5470 From noreply@sourceforge.net Fri Nov 16 21:36:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 13:36:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-482631 ] Integrate Pynche enhancements Message-ID: Bugs item #482631, was opened at 2001-11-16 13:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482631&group_id=5470 Category: Demos and Tools Group: Python 2.2 Status: Open Resolution: None Priority: 6 Submitted By: Barry Warsaw (bwarsaw) Assigned to: Barry Warsaw (bwarsaw) Summary: Integrate Pynche enhancements Initial Comment: Just filing a bug report so I don't forget to integrate Laura Creighton's Pynche enhancements before 2.2 final. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482631&group_id=5470 From noreply@sourceforge.net Fri Nov 16 23:25:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 15:25:07 -0800 Subject: [Python-bugs-list] [ python-Bugs-474919 ] 2.2b1 cannot start remotely (ssh) Message-ID: Bugs item #474919, was opened at 2001-10-25 08:52 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Closed Resolution: Duplicate Priority: 5 Submitted By: Greg Welch (gregwelch) Assigned to: Jack Jansen (jackjansen) Summary: 2.2b1 cannot start remotely (ssh) Initial Comment: Seems possibly related to #466907 This does not seem to occur with 2.2a4, but does with 2.2b1. I tried this on three different Macs, each/all OS X 10.1 (relatively clean installs) with the 10-19-2001 security (and MS IE) patch. All are G4 machines w/ lots of memory. 2.2b1seems to build (configure --with-suffix=.exe), install, and run just fine on all machines when working from a local terminal window. However I see strange behavior when trying to run remotely. 1. If I connect to one of the machines remotely using ssh (e.g. ssh to my dual G4, which is faster than my home machine) and try and start python, I see the following errors: [mac-welch:~/proj/python] welch% python kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorIllegalArgument : CGSNewConnection cannot get connection port kCGErrorInvalidConnection : CGSGetEventPort: Invalid connection [mac-welch:~/proj/python] welch% 2. If I su to root in the same remote session, python starts just fine! If I revert to 2.2a4 it seems to work fine. Thanks (in advance) folks. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-16 15:25 Message: Logged In: NO I have not tried logging in remotely but it seems that the same error comes up when using python through CGI. The circumstances are the same python runs fine when executed via the terminal but if i try to use it via cgi the following error messages come up on the console Nov 17 00:18:52 localhost WindowServer[67]: _CGXNewConnectionPort: Unauthorized user Nov 17 00:18:52 localhost python2.2.exe: kCGErrorIllegalArgument : CGSNewConnection cannot get connection port Nov 17 00:18:52 localhost WindowServer[67]: _CGXNewConnectionPort: Unauthorized user Nov 17 00:18:52 localhost python2.2.exe: kCGErrorIllegalArgument : CGSNewConnection cannot get connection port Nov 17 00:18:52 localhost python2.2.exe: kCGErrorInvalidConnection : CGSGetEventPort: Invalid connection I cannont confirm this but i think this only happened after updating from a clean install of 10.0.3 to 10.1 i had a working version of 2.2a4 on 10.0.3 and after the update neither nor the b1 are working as CGI scripts ---------------------------------------------------------------------- Comment By: Greg Welch (gregwelch) Date: 2001-10-27 06:31 Message: Logged In: YES user_id=324637 I was wrong - apparently this happens w/ 2.2a4 also. There must be some other variable/condition I am missing. While perhaps a "duplicate" of #466907 (same root cause perhaps) running python remotely is probably more likely than running in console mode. I.e. perhaps a slightly higher priority. ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-10-26 05:37 Message: Logged In: YES user_id=45365 This is indeed a duplicate of #466907. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=474919&group_id=5470 From noreply@sourceforge.net Sat Nov 17 06:08:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 16 Nov 2001 22:08:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-482752 ] __getstate__ & __setstate__ ignored Message-ID: Bugs item #482752, was opened at 2001-11-16 22:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __getstate__ & __setstate__ ignored Initial Comment: pickle does not recognize overloaded __getstate__ or __setstate__ methods on subclasses of python built-in types. Instead, the instances' __dict__ attribute is always pickled. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 From noreply@sourceforge.net Sat Nov 17 19:49:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 17 Nov 2001 11:49:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-482871 ] Segmentation Fault with socket lib. Message-ID: Bugs item #482871, was opened at 2001-11-17 11:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482871&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: charles xavier (cqx) Assigned to: Nobody/Anonymous (nobody) Summary: Segmentation Fault with socket lib. Initial Comment: This applies to 2.2 and 2.1.1, I chose 2.2 as the starting group. Change if there's a better way. Here's the code and example. Running on: Linux 2.4.9-13 i686 example: >>> import socket >>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> s.connect( ('localhost', 25) ) >>> s.recv(-1) Segmentation fault ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482871&group_id=5470 From noreply@sourceforge.net Sat Nov 17 22:08:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 17 Nov 2001 14:08:04 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Closed Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-17 14:07 Message: Logged In: NO Are the new docs downloadable from somewhere? thanx --Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-15 16:43 Message: Logged In: NO Barry, can you also do something about the Cookie module, or at least about its documentation? If Cookie is aliased to SmartCookie, I think that needs mention. --Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-15 16:12 Message: Logged In: YES user_id=12800 I have rewritten the pickle documentation, and updated the marshal, cPickle, and copy_reg documentation. I believe all the documentation issues raised here have been fixed. Implementation issues will be pushed to Python 2.3, and the plan is to write a PEP to plan future work. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:39 Message: Logged In: YES user_id=12800 Assigning to Jeremy so he'll remember to forward me Jim's email. Jeremy can ping-pong it to Fred if he wants, and then reassign to me after forwarding the email. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Sat Nov 17 23:41:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 17 Nov 2001 15:41:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Sun Nov 18 00:50:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 17 Nov 2001 16:50:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-482943 ] CGIHTTPServer mod docs need updating Message-ID: Bugs item #482943, was opened at 2001-11-17 16:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482943&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Sandy Norton (sandynorton) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: CGIHTTPServer mod docs need updating Initial Comment: Module reference docs needs to be updated to reflect that post-python-2.0 CGIHTTPServer works with windows. Here's turrent CGIHTTPServer module reference for python 2.2b2: "11.17 CGIHTTPServer -- CGI-capable HTTP request handler Availability: Unix. The CGIHTTPServer module defines a request-handler class, interface compatible with BaseHTTPServer.BaseHTTPRequestHandler and inherits behavior from SimpleHTTPServer.SimpleHTTPRequestHandler but can also run CGI scripts. Note: This module is Unix dependent since it creates the CGI process using os.fork() and os.exec()." Whereas the module docs states the following: ">>> help(CGIHTTPServer) Help on module CGIHTTPServer: NAME CGIHTTPServer - CGI-savvy HTTP Server. FILE c:\engines\python22\lib\cgihttpserver.py DESCRIPTION This module builds on SimpleHTTPServer by implementing GET and POST requests to cgi-bin scripts. If the os.fork() function is not present (e.g. on Windows), os.popen2() is used as a fallback, with slightly altered semantics; if that function is not present either (e.g. on Macintosh), only Python scripts are supported, and they are executed by the current process." ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482943&group_id=5470 From noreply@sourceforge.net Sun Nov 18 01:10:17 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 17 Nov 2001 17:10:17 -0800 Subject: [Python-bugs-list] [ python-Bugs-207608 ] 'python -U' breaks eval/exec Message-ID: Bugs item #207608, was opened at 2000-06-16 16:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=207608&group_id=5470 Category: None Group: None Status: Closed Resolution: Duplicate Priority: 1 Submitted By: Michael Hudson (mwh) Assigned to: M.-A. Lemburg (lemburg) Summary: 'python -U' breaks eval/exec Initial Comment: Freshly built python: [mwh21@atrus build]$ PYTHONSTARTUP= ./python -U 'import site' failed; use -v for traceback Python 1.6a2 (#1, Jun 17 2000, 00:21:58) [GCC 2.95.1 19990816/Linux (release)] on linux2 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam Copyright 1995-2000 Corporation for National Research Initiatives (CNRI) >>> eval("1") Traceback (most recent call last): File "", line 1, in ? TypeError: eval() argument 1 must be string or code object (this is also why the import site fails). exec is also broken. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-17 17:10 Message: Logged In: NO stjfsdgjvvnmbvvvvvv ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2000-06-28 23:38 Message: Copied to Jitterbug on python.org, where it's bug 375. Closed here. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2000-06-26 12:19 Message: eval and exec need to support Unicode objects. This may be hard, and since -U is mostly experimental, I've given it a low priority. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=207608&group_id=5470 From noreply@sourceforge.net Sun Nov 18 03:23:31 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 17 Nov 2001 19:23:31 -0800 Subject: [Python-bugs-list] [ python-Bugs-482871 ] Segmentation Fault with socket lib. Message-ID: Bugs item #482871, was opened at 2001-11-17 11:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482871&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: charles xavier (cqx) >Assigned to: Martin v. Löwis (loewis) Summary: Segmentation Fault with socket lib. Initial Comment: This applies to 2.2 and 2.1.1, I chose 2.2 as the starting group. Change if there's a better way. Here's the code and example. Running on: Linux 2.4.9-13 i686 example: >>> import socket >>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> s.connect( ('localhost', 25) ) >>> s.recv(-1) Segmentation fault ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482871&group_id=5470 From noreply@sourceforge.net Sun Nov 18 06:56:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 17 Nov 2001 22:56:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Closed Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-17 22:56 Message: Logged In: NO Barry - the new docs just went up and they're a big improvement over the old. However the sentence "The issue here is usually not that a class's constructor will get called -- it won't by the unpickler -- but that the class's destructor (i.e. its __del__() method) might get called when the object is garbage collected." isn't correct. There's a flag in the pickle stream that tells the unpickler that the pickler found a __getinitargs__ method at pickling time. If the flag is set in the pickle, then the unpickler calls the class constructor, whether there's a __getinitargs__ method in the class or not. The sample exploit that I posted earlier on, , is an example of an artificially concocted pickle that makes a class constructor get called. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-17 14:07 Message: Logged In: NO Are the new docs downloadable from somewhere? thanx --Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-15 16:43 Message: Logged In: NO Barry, can you also do something about the Cookie module, or at least about its documentation? If Cookie is aliased to SmartCookie, I think that needs mention. --Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-15 16:12 Message: Logged In: YES user_id=12800 I have rewritten the pickle documentation, and updated the marshal, cPickle, and copy_reg documentation. I believe all the documentation issues raised here have been fixed. Implementation issues will be pushed to Python 2.3, and the plan is to write a PEP to plan future work. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:39 Message: Logged In: YES user_id=12800 Assigning to Jeremy so he'll remember to forward me Jim's email. Jeremy can ping-pong it to Fred if he wants, and then reassign to me after forwarding the email. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Sun Nov 18 16:08:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 08:08:20 -0800 Subject: [Python-bugs-list] [ python-Bugs-471893 ] Security review of pickle/marshal docs Message-ID: Bugs item #471893, was opened at 2001-10-16 15:42 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 Category: Extension Modules Group: None Status: Closed Resolution: Fixed Priority: 7 Submitted By: Tim Peters (tim_one) Assigned to: Barry Warsaw (bwarsaw) Summary: Security review of pickle/marshal docs Initial Comment: Paul Rubin points out that the security implications of using marshal and/or pickle aren't clear from the docs. Assigning to Jeremy as he's more sensitive to such issues than I am; maybe Barry would like to get paranoid too . A specific example: the pickle docs say that pickle doesn't support code objects, and "at least this avoids the possibility of smuggling Trojan horses into a program". However, 1) The marshal docs don't mention this vulnerability at all. while 2) The pickle docs don't spell out possible dangers due to things pickle does that marshal doesn't (like importing modules, and running class constructors). ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:08 Message: Logged In: YES user_id=12800 You're right of course. I meant to fix that and forgot. Will do so asap. Glad you like the rest of it, though! :) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-17 22:56 Message: Logged In: NO Barry - the new docs just went up and they're a big improvement over the old. However the sentence "The issue here is usually not that a class's constructor will get called -- it won't by the unpickler -- but that the class's destructor (i.e. its __del__() method) might get called when the object is garbage collected." isn't correct. There's a flag in the pickle stream that tells the unpickler that the pickler found a __getinitargs__ method at pickling time. If the flag is set in the pickle, then the unpickler calls the class constructor, whether there's a __getinitargs__ method in the class or not. The sample exploit that I posted earlier on, , is an example of an artificially concocted pickle that makes a class constructor get called. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-17 14:07 Message: Logged In: NO Are the new docs downloadable from somewhere? thanx --Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-15 16:43 Message: Logged In: NO Barry, can you also do something about the Cookie module, or at least about its documentation? If Cookie is aliased to SmartCookie, I think that needs mention. --Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-15 16:12 Message: Logged In: YES user_id=12800 I have rewritten the pickle documentation, and updated the marshal, cPickle, and copy_reg documentation. I believe all the documentation issues raised here have been fixed. Implementation issues will be pushed to Python 2.3, and the plan is to write a PEP to plan future work. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-13 13:39 Message: Logged In: YES user_id=12800 Assigning to Jeremy so he'll remember to forward me Jim's email. Jeremy can ping-pong it to Fred if he wants, and then reassign to me after forwarding the email. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 22:18 Message: Logged In: NO The find_global variable sounds encouraging and if it fixes this problem, that's great. I'd still feel better if the eval call goes away. Is removing it allowed under the 2.2 feature freeze? It doesn't affect anything visible, so no tests would have to change, etc. Paul ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-12 16:38 Message: Logged In: NO It's possible that the eval is safe, but validating that takes a line by line code review of all the paths that evaling a string can go through. It's also brittle in that maybe someone will change the evaluator (say to support Perl-like interpolation) in a future Python version and not remember to change the unpickler. Something like that has already happened with the Cookie module. My guess is that it happened with the pickle module--the whole point of that quoted string check is that the original pickle implementer didn't trust the input. The stuff about unpickling class instances was added later (maybe by another person) without remembering the security issue. Using eval this way is like storing a vat of cyanide in your child's bedroom. Sure, maybe if you check the seals and locks on the vat carefully enough, you can convince yourself that your child won't be able to get to the cyanide. But wouldn't you feel safer just not having the vat there at all? That's basic safety-consciousness. Security consciousness works the same way. Try to keep dangerous ingredients and attackers as far away from each other as possible. Paul ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-12 10:52 Message: Logged In: YES user_id=31435 Why are people (Paul, Jeremy) concerned about eval'ing strings? cPickle and pickle both check that they're properly quoted, and this isn't sh or Perl: Python has no "dynamic" gimmicks buried in string literals. All eval'ing a string literal can do is produce a binary blob via interpeting simple escape sequences. They're like C strings this way -- maybe we'll run out of memory, but that's it. I would agree that Python should be refactored internally to supply a clean function for changing string literals into binary blobs, but that would be for clarity and efficiency, not security. ---------------------------------------------------------------------- Comment By: Jim Fulton (dcjim) Date: 2001-11-12 10:14 Message: Logged In: YES user_id=73023 It sounds like there are some documentation bugs: - The security ramifications are not discussed, nor are the remedies. - The cPickle module isn't documented very well. I submitted some documentation a long time ago that never got incorporated AFAIK. I wish I still had it. :) - cPickle has a feature for turning off instance support and for restricting which classes can be unpickled. You can set the find_global attribute on a cPickle.Unpickler. The find_global attribute can be a function or None. If it is None, then no instances can be unpickled. If it is a function, then it should accept a module and name and return the corresponding global. It is responsible for looking up the global and can raise an error to prevent a global's use. See the ZEO storage server implementation for an example of using this hook. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-11 22:33 Message: Logged In: NO I like marshal and think it's much cleaner than pickle. I'd like to see its format cleaned up and documented so it can be a portable transport format. Even the recursion issue isn't so compelling--there's only a danger on marshalling, not unmarshalling, and the application has control over what it marshals. So I think the idea of documenting marshal (bug #467384) should be kept alive. However, it shouldn't be changed in 2.2. If the Cookie class is still really aliased by default to SmartCookie, that's a bad bug and should definitely be fixed ASAP. The Cookie docs clearly say not to use SmartCookie or SerialCookie. In fact I think SmartCookie and SerialCookie should be disabled altogether. Any applications using them on the open internet have a security but and should be fixed immediately. Here's a possible temporary approach to fixing SmartCookie and SerialCookie: append a cryptographic message authentication code (MAC) to each pickled cookie. That means the application has some secret string K as a configuration parameter (it should be 10 or more random 8-bit characters, or 20 or more random hex digits, etc). Then a smart cookie would be a pickle p, followed by SHA(p+K). The Cookie module would validate the MAC before calling unpickle, and raise an exception if the MAC wasn't valid. The security of this scheme rests on K being kept secret from attackers, which is generally not an easy thing to manage, but it's something to think about. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-10 14:07 Message: Logged In: YES user_id=31392 I don't think we should be doing anything about marshal. Maybe we should name in pyclib or something <0.9 wink>. It works fine for .pyc files, but I don't see a reason for it to do anymore than is necessary for that purpose. I think the notion of an unpickler that only handles builtin datatypes is the most attractive option you offer, but Paul has a good point about eval for strings. (It currently has some hacks that address specific exploits, but I doubt they are sufficient.) I also wonder how hard it is to handle builtin types and avoid subclasses of builtin types. If there are any changes to pickle, I think we need to be careful about how it is described. If we claim that an unpickler is safe for untrusted pickles, we've made a fairly strong claim. I still think such a design change requires a PEP that includes some requirements and use cases and a thorough analysis of potential exploits. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-10 11:42 Message: Logged In: NO See bug #467384 for discussion about marshal. Besides the recursion issue, marshal's format is explicitly undocumented and subject to change--you can't rely on it to interoperate between two different Python versions, so it's no good as an RPC serializer. The format has kludges (e.g. the representation of long ints) that make it undesirable to freeze and document it and force future versions to be backward compatible. Adding a pickle.loads flag to prevent instance unpickling isn't perfect but is probably the best alternative on your list. Perhaps the flag can have a value that allows unpickling the instances by restoring the instance attributes rather than calling the initializer. That's not always the right way to unpickle an instance (that's why the unpickler no longer works that way) but it's good enough a lot of the time. There's another issue with pickle/cPickle which is that they unpickle quoted strings by evaling them. This is scary. While I don't see an immediate exploit, I also haven't examined the 1000's of lines of code I'd need to examine to convince myself that there's NOT an exploit. I think the unpickler should be changed to never call eval but just parse the string as it needs to. Guido seemed to think pickle might have other possible exploits. I don't know what he had in mind but before declaring it safe for untrusted data I think it needs to be gone over with a fine toothed comb. Paul ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-10 09:37 Message: Logged In: YES user_id=12800 I'm going to agree with Paul that this is a problem needing fixing, however there are really several issues. 1. Cookie module makes it too easy to code exploits. Cookie exports a class, also called Cookie, which is aliased to SmartCookie, so that a naive program will simply pass cookie data to Cookie.Cookie() and you're screwed. So, Cookie module's defaults should be for more security rather than less, and Cookie.Cookie should be aliased to SimpleCookie instead. 2. There is no built-in safe mechanism for de-serializing data from untrusted sources. You can't use pickle without overloading a "magic" method. You can't use cPickle because you can't do the overloading trick. You can't use marshal because it isn't bulletproof against recursive datastructures. So how /do/ you do it? I think it just may be serious enough to deal with in Python 2.2, and I volunteer to address it (so I'll steal this bug report). Without looking at the code, or the level of effort necessary, I would make the following suggestions: 1. Add to the public interface of pickle and cPickle, a flag that either disables the unpickling of instances altogether, or disables calling any code with unpickled data, e.g. constructors. 2. Fix marshal to be bulletproof against recursive datastructures. 3. Update the docs for both pickle/cPickle and marshal to explain how to safely write de-serializers of untrusted strings. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-09 01:21 Message: Logged In: NO Well, Guido and Tim agree with you that it's not a pickle bug. I still feel it is one, because its docs currently make you think you can securely load untrusted pickles, and because it's a natural, non-obscure thing to want to do (see pyro and the cookie module), but whatever. If it's not a code bug then I feel it's a significant functionality shortcoming in the python library. Pyro uses pickle to serialize data for RPC calls over the internet. A malicious client could make a hostile pickle take over the server. The cookie module lets web applications store user session state in browser cookies. Its SerialCookie and SmartCookie classes let you put arbitrary Python objects into the user session, and serializes them when pickle. Again, a malicious client can make a hostile pickle, send it in a cookie header to the http server, and take over the server when the application unpickles the cookie. The current documentation for the pickle module makes it very clear to me that the doc writer thought it was safe to unpickle untrusted cookies. If pickle wasn't designed for that, then there was a communication failure between the designer and the doc writer. Yes, I'm willing to help with a PEP for fixing this situation. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-08 09:37 Message: Logged In: YES user_id=31392 I don't think of the issue you describe as a bug in the code. You're suggesting a new feature for pickle. As far as I can tell, the original design requirements for pickle did not include the ability to securely load a pickle from an untrusted source. It may be a legitimate feature request, but it's too late to make it into Python 2.2. I suggest we look at the design issues for Python 2.3 and decide if it's a feature we want to support. I imagine a PEP may be necessary to lay out the issues and the solution. Do you want to have a hand in that PEP? I still don't understand what it means that Pyro and cookie were bit by a bug. It sounds like they were using pickle in ways that pickle was not intended to support. A careful analysis of how those two applications use pickle would be helpful for generating requirements. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 15:54 Message: Logged In: NO IMO it's a code bug that you can't unpickle strings from untrusted sources. Pyro and the cookie module are examples of programs that got bitten by this bug. Whether it's really a bug is a matter of opinion--I had a big email exchange with Guido and Tim about it, and they felt it was enough to fix the pickle documentation. Pickle has the same problem as cPickle, but with pickle you can subclass the pickler and override the method that unpickles class objects, and work around the (IMO) bug. The workaround doesn't help cPickle since cPickle can't be subclassed. See bug #467384 for some related discussion. Paul ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-07 14:02 Message: Logged In: YES user_id=31392 What's the code bug? Your last message has a lot of gloom and doom , but I'm not sure what specific problem you'd like to see fixed. Are you saying that something needs to be fixed in cPickle and not in pickle? ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-07 12:08 Message: Logged In: NO Irmen de Jong points out that the standard cookie module uses pickling for serial and smart cookies. The 2.1.1 cookie module docs explicitly say not to use those classes because of the security hole--that they're provided for backward compatibility only (but with what?! Any server that uses those classes on open ports needs to be changed right away). Irmen's library, http://pyro.sourceforge.net, also uses unpickle insecurely (he's aware of this now and is figuring out a fix). IMO this is really a code bug rather than a documentation bug, and should be fixed in the code rather than the docs. Documenting the bug rather than fixing it leaves a deficiency in the Python library: obvious uses of pickling, like Pyro and the cookie module, can't be implemented using cPickle and have to resort to a slower Python deserializer, or use marshal and have possible compatibility problems between versions (and not be able to serialize class instances). Paul ---------------------------------------------------------------------- Comment By: paul rubin (phr) Date: 2001-10-16 16:06 Message: Logged In: YES user_id=72053 Certainly anyone unserializing potentially malicious data with pickle, marshal, or anything else, should check the results before doing anything dangerous with them (like executing code). However, unpickle can potentially do damage before it even returns, by creating loading modules and creating initialized class instances. So pickle.loads should never be used on untrusted strings, except possibly in a rexec wrapper (as proposed by Tim). Another possibility (also by Tim) is to override the load_inst method of the Pickler class, though I don't think you can do that for cPickle. A sample exploit for unpickle can be found at . Unpickling the test string runs penguin.__init__ contrary to the doc's saying no initialization unless there's a __getinitargs__ method in the class def. The "exploding penguin" class is artificial, but applications are vulnerable if there's an unsafe constructor anywhere in any class of the application or in the python library (example: the NNTP constructor opens an IP connection to an arbitrary address, so a malicious imported string can send a message through your firewall when you import it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=471893&group_id=5470 From noreply@sourceforge.net Sun Nov 18 16:46:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 08:46:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:46 Message: Logged In: YES user_id=12800 Tell me what strings time.ctime(0) and Utils.formatdate(now) returns and I'll adjust the test case. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Sun Nov 18 18:38:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 10:38:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-483126 ] cgitb does not recover Message-ID: Bugs item #483126, was opened at 2001-11-18 10:38 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483126&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Dan Grassi (dgrassi) Assigned to: Nobody/Anonymous (nobody) Summary: cgitb does not recover Initial Comment: chitb does not recover when sys.stdout has been resigned. The fix is simple, set sys.stdout to sys.__stdout__ in cgitb.handle right after the import of sys. line 160: def handle(self, info=None): import sys + sys.stdout = sys.__stdout__ print reset() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483126&group_id=5470 From noreply@sourceforge.net Sun Nov 18 22:44:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 14:44:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 14:44 Message: Logged In: YES user_id=45365 Here is the output with the relevant numbers: Python 2.2b2 (#114, Nov 18 2001, 23:23:26) [CW PPC GUSI2 GC] on mac Type "copyright", "credits" or "license" for more information. >>> import time >>> now = time.time() >>> time.ctime(now) 'Sun Nov 18 23:23:56 2001' >>> time.ctime(0) 'Fri Jan 1 00:00:00 1904' >>> import email >>> email.Utils.formatdate(now) 'Sun, 18 Nov 2001 22:23:56 -0000' ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:46 Message: Logged In: YES user_id=12800 Tell me what strings time.ctime(0) and Utils.formatdate(now) returns and I'll adjust the test case. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Sun Nov 18 23:01:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 15:01:04 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:01 Message: Logged In: YES user_id=12800 My bad Jack. I meant, what does Utils.formatdate(1005327232.109884) return? (see test_formatdate) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 14:44 Message: Logged In: YES user_id=45365 Here is the output with the relevant numbers: Python 2.2b2 (#114, Nov 18 2001, 23:23:26) [CW PPC GUSI2 GC] on mac Type "copyright", "credits" or "license" for more information. >>> import time >>> now = time.time() >>> time.ctime(now) 'Sun Nov 18 23:23:56 2001' >>> time.ctime(0) 'Fri Jan 1 00:00:00 1904' >>> import email >>> email.Utils.formatdate(now) 'Sun, 18 Nov 2001 22:23:56 -0000' ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:46 Message: Logged In: YES user_id=12800 Tell me what strings time.ctime(0) and Utils.formatdate(now) returns and I'll adjust the test case. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Sun Nov 18 23:08:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 15:08:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 15:08 Message: Logged In: YES user_id=45365 Sorry, it returns Sat, 09 Nov 1935 16:33:52 -0000. But: I hope your fix isn't going to if "if os.name == 'mac': compare to another value"? I could have come up with that one:-) I would assume that you're trying to test that a given date/time is rendered as such, in which case I would suggest using a time tuple, which contains exact numbers for year and everything, and contains the original timezone/dst flag as well. But: (why I didn't fix it myself) I'm not sure that this is actually what you're testing... ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:01 Message: Logged In: YES user_id=12800 My bad Jack. I meant, what does Utils.formatdate(1005327232.109884) return? (see test_formatdate) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 14:44 Message: Logged In: YES user_id=45365 Here is the output with the relevant numbers: Python 2.2b2 (#114, Nov 18 2001, 23:23:26) [CW PPC GUSI2 GC] on mac Type "copyright", "credits" or "license" for more information. >>> import time >>> now = time.time() >>> time.ctime(now) 'Sun Nov 18 23:23:56 2001' >>> time.ctime(0) 'Fri Jan 1 00:00:00 1904' >>> import email >>> email.Utils.formatdate(now) 'Sun, 18 Nov 2001 22:23:56 -0000' ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:46 Message: Logged In: YES user_id=12800 Tell me what strings time.ctime(0) and Utils.formatdate(now) returns and I'll adjust the test case. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Sun Nov 18 23:13:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 15:13:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:13 Message: Logged In: YES user_id=12800 Actually, I had planned to test time.ctime(0), but you're probably right that it would be better to test time.gmtime(0). The first part of the test is mostly testing that a numeric timezone is returned instead of a GMT timezone. The second half attempts to test that the numeric timezone for the local time matches what's expected given .daylight, .timezone, and .altzone. I'll check in a patch (I think I have everything I need), but you may need to adjust it for your output. 09-Nov-1935 doesn't make much sense given that the float time is (much) less than 66 years ago. :) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 15:08 Message: Logged In: YES user_id=45365 Sorry, it returns Sat, 09 Nov 1935 16:33:52 -0000. But: I hope your fix isn't going to if "if os.name == 'mac': compare to another value"? I could have come up with that one:-) I would assume that you're trying to test that a given date/time is rendered as such, in which case I would suggest using a time tuple, which contains exact numbers for year and everything, and contains the original timezone/dst flag as well. But: (why I didn't fix it myself) I'm not sure that this is actually what you're testing... ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:01 Message: Logged In: YES user_id=12800 My bad Jack. I meant, what does Utils.formatdate(1005327232.109884) return? (see test_formatdate) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 14:44 Message: Logged In: YES user_id=45365 Here is the output with the relevant numbers: Python 2.2b2 (#114, Nov 18 2001, 23:23:26) [CW PPC GUSI2 GC] on mac Type "copyright", "credits" or "license" for more information. >>> import time >>> now = time.time() >>> time.ctime(now) 'Sun Nov 18 23:23:56 2001' >>> time.ctime(0) 'Fri Jan 1 00:00:00 1904' >>> import email >>> email.Utils.formatdate(now) 'Sun, 18 Nov 2001 22:23:56 -0000' ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:46 Message: Logged In: YES user_id=12800 Tell me what strings time.ctime(0) and Utils.formatdate(now) returns and I'll adjust the test case. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Mon Nov 19 02:33:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 18:33:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-483231 ] email.Utils.formatdate(): localtime bug Message-ID: Bugs item #483231, was opened at 2001-11-18 18:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Nobody/Anonymous (nobody) Summary: email.Utils.formatdate(): localtime bug Initial Comment: email.Utils.formatdate(localtime=1) does not produce a correct UTC offset for "uneven" timezones (e.g, "+1030", "+0530") In those cases, it rounds the offset down to "+1000", and "+0500". In the example below, you can see that python produces the wrong UTC offset once I set my TZ to "Australia/Adelaide", a timezone whose offset should be "+1030". $ date Sun Nov 18 19:29:22 MST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Sun, 18 Nov 2001 19:29:22 -0700 $ export TZ="Australia/Adelaide" $ date Mon Nov 19 12:59:35 CST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Mon, 19 Nov 2001 12:59:35 +1000 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 From noreply@sourceforge.net Mon Nov 19 02:50:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 18:50:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-483236 ] subclass of int Message-ID: Bugs item #483236, was opened at 2001-11-18 18:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483236&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: subclass of int Initial Comment: a in the following example should be 220 >>> class ticks(int): ... __slots__ = ['clk'] ... def __init__(self,dd): ... dd*=self.clk ... int.__init__(self,dd) ... >>> ticks.clk=11 >>> a=ticks(20) >>> a 20 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483236&group_id=5470 From noreply@sourceforge.net Mon Nov 19 02:54:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 18:54:21 -0800 Subject: [Python-bugs-list] [ python-Bugs-483231 ] email.Utils.formatdate(): localtime bug Message-ID: Bugs item #483231, was opened at 2001-11-18 18:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Nobody/Anonymous (nobody) Summary: email.Utils.formatdate(): localtime bug Initial Comment: email.Utils.formatdate(localtime=1) does not produce a correct UTC offset for "uneven" timezones (e.g, "+1030", "+0530") In those cases, it rounds the offset down to "+1000", and "+0500". In the example below, you can see that python produces the wrong UTC offset once I set my TZ to "Australia/Adelaide", a timezone whose offset should be "+1030". $ date Sun Nov 18 19:29:22 MST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Sun, 18 Nov 2001 19:29:22 -0700 $ export TZ="Australia/Adelaide" $ date Mon Nov 19 12:59:35 CST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Mon, 19 Nov 2001 12:59:35 +1000 ---------------------------------------------------------------------- >Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-18 18:54 Message: Logged In: YES user_id=85984 Just to followup with some more info: I forgot to mention I'm running Python 2.2b2 on FreeBSD 4.4-RELEASE. Also, when investigating this problem, don't forget timezones with negative fractional UTC offsets (e.g, -0330) in addition to positive. Such is the case when TZ is "Canada/Newfoundland" or "America/St_Johns" for example. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 From noreply@sourceforge.net Mon Nov 19 05:17:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 21:17:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-482943 ] CGIHTTPServer mod docs need updating Message-ID: Bugs item #482943, was opened at 2001-11-17 16:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482943&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Sandy Norton (sandynorton) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: CGIHTTPServer mod docs need updating Initial Comment: Module reference docs needs to be updated to reflect that post-python-2.0 CGIHTTPServer works with windows. Here's turrent CGIHTTPServer module reference for python 2.2b2: "11.17 CGIHTTPServer -- CGI-capable HTTP request handler Availability: Unix. The CGIHTTPServer module defines a request-handler class, interface compatible with BaseHTTPServer.BaseHTTPRequestHandler and inherits behavior from SimpleHTTPServer.SimpleHTTPRequestHandler but can also run CGI scripts. Note: This module is Unix dependent since it creates the CGI process using os.fork() and os.exec()." Whereas the module docs states the following: ">>> help(CGIHTTPServer) Help on module CGIHTTPServer: NAME CGIHTTPServer - CGI-savvy HTTP Server. FILE c:\engines\python22\lib\cgihttpserver.py DESCRIPTION This module builds on SimpleHTTPServer by implementing GET and POST requests to cgi-bin scripts. If the os.fork() function is not present (e.g. on Windows), os.popen2() is used as a fallback, with slightly altered semantics; if that function is not present either (e.g. on Macintosh), only Python scripts are supported, and they are executed by the current process." ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-18 21:17 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/libcgihttp.tex revisions 1.7 and 1.6.2.1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482943&group_id=5470 From noreply@sourceforge.net Mon Nov 19 05:24:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 21:24:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-482943 ] CGIHTTPServer mod docs need updating Message-ID: Bugs item #482943, was opened at 2001-11-17 16:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482943&group_id=5470 Category: Documentation Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Sandy Norton (sandynorton) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: CGIHTTPServer mod docs need updating Initial Comment: Module reference docs needs to be updated to reflect that post-python-2.0 CGIHTTPServer works with windows. Here's turrent CGIHTTPServer module reference for python 2.2b2: "11.17 CGIHTTPServer -- CGI-capable HTTP request handler Availability: Unix. The CGIHTTPServer module defines a request-handler class, interface compatible with BaseHTTPServer.BaseHTTPRequestHandler and inherits behavior from SimpleHTTPServer.SimpleHTTPRequestHandler but can also run CGI scripts. Note: This module is Unix dependent since it creates the CGI process using os.fork() and os.exec()." Whereas the module docs states the following: ">>> help(CGIHTTPServer) Help on module CGIHTTPServer: NAME CGIHTTPServer - CGI-savvy HTTP Server. FILE c:\engines\python22\lib\cgihttpserver.py DESCRIPTION This module builds on SimpleHTTPServer by implementing GET and POST requests to cgi-bin scripts. If the os.fork() function is not present (e.g. on Windows), os.popen2() is used as a fallback, with slightly altered semantics; if that function is not present either (e.g. on Macintosh), only Python scripts are supported, and they are executed by the current process." ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-18 21:24 Message: Logged In: YES user_id=3066 Checked in equivalent fix in Doc/lib/libcgihttp.tex revision 1.5.4.1. Version 1.6.2.1 was a mistake; it should not have been checked in. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-18 21:17 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/libcgihttp.tex revisions 1.7 and 1.6.2.1. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482943&group_id=5470 From noreply@sourceforge.net Mon Nov 19 07:24:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 18 Nov 2001 23:24:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 23:24 Message: Logged In: YES user_id=12800 In fact the epoch *must* be tested with gmtime(0). It would be good if I read the docs, eh? I'll fix that. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:13 Message: Logged In: YES user_id=12800 Actually, I had planned to test time.ctime(0), but you're probably right that it would be better to test time.gmtime(0). The first part of the test is mostly testing that a numeric timezone is returned instead of a GMT timezone. The second half attempts to test that the numeric timezone for the local time matches what's expected given .daylight, .timezone, and .altzone. I'll check in a patch (I think I have everything I need), but you may need to adjust it for your output. 09-Nov-1935 doesn't make much sense given that the float time is (much) less than 66 years ago. :) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 15:08 Message: Logged In: YES user_id=45365 Sorry, it returns Sat, 09 Nov 1935 16:33:52 -0000. But: I hope your fix isn't going to if "if os.name == 'mac': compare to another value"? I could have come up with that one:-) I would assume that you're trying to test that a given date/time is rendered as such, in which case I would suggest using a time tuple, which contains exact numbers for year and everything, and contains the original timezone/dst flag as well. But: (why I didn't fix it myself) I'm not sure that this is actually what you're testing... ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:01 Message: Logged In: YES user_id=12800 My bad Jack. I meant, what does Utils.formatdate(1005327232.109884) return? (see test_formatdate) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 14:44 Message: Logged In: YES user_id=45365 Here is the output with the relevant numbers: Python 2.2b2 (#114, Nov 18 2001, 23:23:26) [CW PPC GUSI2 GC] on mac Type "copyright", "credits" or "license" for more information. >>> import time >>> now = time.time() >>> time.ctime(now) 'Sun Nov 18 23:23:56 2001' >>> time.ctime(0) 'Fri Jan 1 00:00:00 1904' >>> import email >>> email.Utils.formatdate(now) 'Sun, 18 Nov 2001 22:23:56 -0000' ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:46 Message: Logged In: YES user_id=12800 Tell me what strings time.ctime(0) and Utils.formatdate(now) returns and I'll adjust the test case. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Mon Nov 19 10:42:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 02:42:07 -0800 Subject: [Python-bugs-list] [ python-Bugs-482871 ] Segmentation Fault with socket lib. Message-ID: Bugs item #482871, was opened at 2001-11-17 11:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482871&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: charles xavier (cqx) Assigned to: Martin v. Löwis (loewis) Summary: Segmentation Fault with socket lib. Initial Comment: This applies to 2.2 and 2.1.1, I chose 2.2 as the starting group. Change if there's a better way. Here's the code and example. Running on: Linux 2.4.9-13 i686 example: >>> import socket >>> s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) >>> s.connect( ('localhost', 25) ) >>> s.recv(-1) Segmentation fault ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-19 02:42 Message: Logged In: YES user_id=21627 Thanks for the report. Fixed in socketmodule.c 1.196. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482871&group_id=5470 From noreply@sourceforge.net Mon Nov 19 14:18:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 06:18:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-479967 ] Python/C API manual has no index section Message-ID: Bugs item #479967, was opened at 2001-11-09 04:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 7 Submitted By: Thomas Heller (theller) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Python/C API manual has no index section Initial Comment: In the development docs, the Python/C API manual the index section is missing. ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2001-11-19 06:18 Message: Logged In: YES user_id=86307 The docs included with the Windows 2.2b2 distribution only have indexes (genindex.html) for the lib and mac subdirectories. The indexes for the C API and Python Reference Manual are missing (I assume this is the same bug as reported here). ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-09 06:43 Message: Logged In: YES user_id=3066 Raised the severity of this -- this *must* be fixed! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 From noreply@sourceforge.net Mon Nov 19 15:50:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 07:50:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on assignment in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Dallas T. Johnston (elaias) Assigned to: Nobody/Anonymous (nobody) Summary: crash on assignment in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Mon Nov 19 16:26:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 08:26:54 -0800 Subject: [Python-bugs-list] [ python-Bugs-483231 ] email.Utils.formatdate(): localtime bug Message-ID: Bugs item #483231, was opened at 2001-11-18 18:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) >Assigned to: Barry Warsaw (bwarsaw) Summary: email.Utils.formatdate(): localtime bug Initial Comment: email.Utils.formatdate(localtime=1) does not produce a correct UTC offset for "uneven" timezones (e.g, "+1030", "+0530") In those cases, it rounds the offset down to "+1000", and "+0500". In the example below, you can see that python produces the wrong UTC offset once I set my TZ to "Australia/Adelaide", a timezone whose offset should be "+1030". $ date Sun Nov 18 19:29:22 MST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Sun, 18 Nov 2001 19:29:22 -0700 $ export TZ="Australia/Adelaide" $ date Mon Nov 19 12:59:35 CST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Mon, 19 Nov 2001 12:59:35 +1000 ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-18 18:54 Message: Logged In: YES user_id=85984 Just to followup with some more info: I forgot to mention I'm running Python 2.2b2 on FreeBSD 4.4-RELEASE. Also, when investigating this problem, don't forget timezones with negative fractional UTC offsets (e.g, -0330) in addition to positive. Such is the case when TZ is "Canada/Newfoundland" or "America/St_Johns" for example. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 From noreply@sourceforge.net Mon Nov 19 16:31:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 08:31:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-483231 ] email.Utils.formatdate(): localtime bug Message-ID: Bugs item #483231, was opened at 2001-11-18 18:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Barry Warsaw (bwarsaw) Summary: email.Utils.formatdate(): localtime bug Initial Comment: email.Utils.formatdate(localtime=1) does not produce a correct UTC offset for "uneven" timezones (e.g, "+1030", "+0530") In those cases, it rounds the offset down to "+1000", and "+0500". In the example below, you can see that python produces the wrong UTC offset once I set my TZ to "Australia/Adelaide", a timezone whose offset should be "+1030". $ date Sun Nov 18 19:29:22 MST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Sun, 18 Nov 2001 19:29:22 -0700 $ export TZ="Australia/Adelaide" $ date Mon Nov 19 12:59:35 CST 2001 $ python2.2 -c 'import email;print email.Utils.formatdate(localtime=1)' Mon, 19 Nov 2001 12:59:35 +1000 ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-19 08:31 Message: Logged In: YES user_id=12800 Fixed in Lib/email/Utils.py 1.7 ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-18 18:54 Message: Logged In: YES user_id=85984 Just to followup with some more info: I forgot to mention I'm running Python 2.2b2 on FreeBSD 4.4-RELEASE. Also, when investigating this problem, don't forget timezones with negative fractional UTC offsets (e.g, -0330) in addition to positive. Such is the case when TZ is "Canada/Newfoundland" or "America/St_Johns" for example. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483231&group_id=5470 From noreply@sourceforge.net Mon Nov 19 16:32:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 08:32:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-482932 ] test_email makes assumptions about epoch Message-ID: Bugs item #482932, was opened at 2001-11-17 15:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 Category: Python Library Group: None >Status: Closed >Resolution: Fixed Priority: 8 Submitted By: Jack Jansen (jackjansen) Assigned to: Barry Warsaw (bwarsaw) Summary: test_email makes assumptions about epoch Initial Comment: test_email makes assumptions about the time.time() epoch being the Unix-standard of Jan 1, 1970; in method test_formatdate(). This test fails on the Mac, where the epoch is Jan 1, 1904. As I'm unsure of what the test actually tests I don't know whether the test is wrong or whether Utils.formatdate() is wrong. If this can be fixed quickly please let me know and I'll incorporate the fix in MacPython 2.2b2, otherwise I'll add a note to the readme. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-19 08:32 Message: Logged In: YES user_id=12800 Fixed in test_email.py 1.20 ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 23:24 Message: Logged In: YES user_id=12800 In fact the epoch *must* be tested with gmtime(0). It would be good if I read the docs, eh? I'll fix that. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:13 Message: Logged In: YES user_id=12800 Actually, I had planned to test time.ctime(0), but you're probably right that it would be better to test time.gmtime(0). The first part of the test is mostly testing that a numeric timezone is returned instead of a GMT timezone. The second half attempts to test that the numeric timezone for the local time matches what's expected given .daylight, .timezone, and .altzone. I'll check in a patch (I think I have everything I need), but you may need to adjust it for your output. 09-Nov-1935 doesn't make much sense given that the float time is (much) less than 66 years ago. :) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 15:08 Message: Logged In: YES user_id=45365 Sorry, it returns Sat, 09 Nov 1935 16:33:52 -0000. But: I hope your fix isn't going to if "if os.name == 'mac': compare to another value"? I could have come up with that one:-) I would assume that you're trying to test that a given date/time is rendered as such, in which case I would suggest using a time tuple, which contains exact numbers for year and everything, and contains the original timezone/dst flag as well. But: (why I didn't fix it myself) I'm not sure that this is actually what you're testing... ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 15:01 Message: Logged In: YES user_id=12800 My bad Jack. I meant, what does Utils.formatdate(1005327232.109884) return? (see test_formatdate) ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-18 14:44 Message: Logged In: YES user_id=45365 Here is the output with the relevant numbers: Python 2.2b2 (#114, Nov 18 2001, 23:23:26) [CW PPC GUSI2 GC] on mac Type "copyright", "credits" or "license" for more information. >>> import time >>> now = time.time() >>> time.ctime(now) 'Sun Nov 18 23:23:56 2001' >>> time.ctime(0) 'Fri Jan 1 00:00:00 1904' >>> import email >>> email.Utils.formatdate(now) 'Sun, 18 Nov 2001 22:23:56 -0000' ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-18 08:46 Message: Logged In: YES user_id=12800 Tell me what strings time.ctime(0) and Utils.formatdate(now) returns and I'll adjust the test case. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482932&group_id=5470 From noreply@sourceforge.net Mon Nov 19 17:18:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 09:18:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on assignment in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None >Priority: 4 Submitted By: Dallas T. Johnston (elaias) Assigned to: Nobody/Anonymous (nobody) Summary: crash on assignment in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Mon Nov 19 17:45:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 09:45:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on unbounded recursion in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 4 Submitted By: Dallas T. Johnston (elaias) Assigned to: Nobody/Anonymous (nobody) >Summary: crash on unbounded recursion in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- >Comment By: Dallas T. Johnston (elaias) Date: 2001-11-19 09:45 Message: Logged In: YES user_id=109241 Sorry, I thought that this was obvious enough for the person responsible that I didn't have to explain. Thanks for the correction though. And I would say that a segfault in any program is a problem, especially an interpreter. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Tue Nov 20 02:41:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 19 Nov 2001 18:41:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-483653 ] bsddb is not built on RedHat 7.2 Message-ID: Bugs item #483653, was opened at 2001-11-19 18:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bsddb is not built on RedHat 7.2 Initial Comment: Build of bsddb fails under RedHat 7.2 because the appropriate library to link against is libdb-3.2.so which is not checked for by the setup.py script. The fix is trivial and attached along with this message. raj ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 From noreply@sourceforge.net Tue Nov 20 14:15:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 06:15:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-483789 ] another *? greedy bug Message-ID: Bugs item #483789, was opened at 2001-11-20 06:15 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483789&group_id=5470 Category: Regular Expressions Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Bastian Kleineidam (calvin) Assigned to: Fredrik Lundh (effbot) Summary: another *? greedy bug Initial Comment: Hm, it took me one hour debugging until I reached the regex level of my application :) Python 2.2b2 (#26, Nov 16 2001, 11:44:11) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> re.search(r"a[^>]*?b", "a>b") ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483789&group_id=5470 From noreply@sourceforge.net Tue Nov 20 14:20:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 06:20:33 -0800 Subject: [Python-bugs-list] [ python-Bugs-438164 ] Format operator % fails with unicode Message-ID: Bugs item #438164, was opened at 2001-07-03 00:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=438164&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Chema Cortés (chemacortes) Assigned to: M.-A. Lemburg (lemburg) Summary: Format operator % fails with unicode Initial Comment: A dictionary with unicode keys raise an error when it is applied to a string throught the format operator %. Here is the ofending code: >>> "%(año)d" % { u"año":2001 } # KeyError: año This fails because u"año" and "año" cannot be compared: byte strings and Unicode strings are comparable only if the byte string is ASCII. > >>> u"%(año)4d" % { u"año":2001 } # KeyError: año This fails because the key is converted to UTF-8 before lookup. Must be add the unicode case into the format operator % ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-20 06:20 Message: Logged In: YES user_id=38388 The first case cannot be fixed since there's no way to magically know that the lookup string is in Latin-1. The second is slightly different: this was from the very beginning of the Unicode implementation where we used UTF-8 as default encoding. I'll have to think about this some more, but I believe that we should drop the conversion altogether and use the Unicode object for the lookup rather than converting it to a string. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:02 Message: Logged In: YES user_id=6380 Wow, that's some vacation! :-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-08-16 03:47 Message: Logged In: YES user_id=38388 I'll look into this after I'm back from vacation on the 10.09. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=438164&group_id=5470 From noreply@sourceforge.net Tue Nov 20 14:48:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 06:48:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-483802 ] How to Crash Python In 3 Easy Steps :) Message-ID: Bugs item #483802, was opened at 2001-11-20 06:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483802&group_id=5470 Category: Python Interpreter Core Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Andrew Bennetts (spiv) Assigned to: Nobody/Anonymous (nobody) Summary: How to Crash Python In 3 Easy Steps :) Initial Comment: This is easy to demonstrate: Python 2.1.1 (#1, Aug 1 2001, 23:36:34) [GCC 2.95.4 20010319 (Debian prerelease)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import sys >>> del sys.modules['__builtin__'] >>> int('1') Segmentation fault I don't have a copy of any of the 2.2 betas, so I haven't tested it in those. Apologies if this bug has already been fixed. I realise this is a silly thing to even try to do... I was just wondering what would happen. 1.5.2 and 2.0.1 appear to be fine, incidentally. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483802&group_id=5470 From noreply@sourceforge.net Tue Nov 20 14:53:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 06:53:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on unbounded recursion in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 4 Submitted By: Dallas T. Johnston (elaias) Assigned to: Nobody/Anonymous (nobody) Summary: crash on unbounded recursion in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- Comment By: Andrew Bennetts (spiv) Date: 2001-11-20 06:53 Message: Logged In: YES user_id=50945 I agree that segfaults like this are a problem; this is a potential security problem because it means that untrusted code can crash the interpreter, even if you try to use something like rexec. ---------------------------------------------------------------------- Comment By: Dallas T. Johnston (elaias) Date: 2001-11-19 09:45 Message: Logged In: YES user_id=109241 Sorry, I thought that this was obvious enough for the person responsible that I didn't have to explain. Thanks for the correction though. And I would say that a segfault in any program is a problem, especially an interpreter. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Tue Nov 20 14:55:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 06:55:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-483805 ] Minor description problem Message-ID: Bugs item #483805, was opened at 2001-11-20 06:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483805&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Minor description problem Initial Comment: Subject: Re: Mutable strings Date: Sun, 18 Nov 2001 22:45:30 -0500 From: "Fred L. Drake, Jr." To: "Colin J. Williams" CC: Python-docs@python.org References: 1 Colin J. Williams writes: > Section 7.7 of the library (mmap) refers to mutable strings. > > At first glance, this is jarring. We learn early on that strings are > immutable. Please file a bug report at SourceForge for this so I don't lose track of it; I'm sure it won't be a difficult fix, but I can't work on it right now. http://sourceforge.net/projects/python/ > Finally, a little grumble about Active State's conversion of the > docs from HTML to the Microsoft world. Among other things, the > links on the About page force one to use the MS Mailer. My default > browser/email facility is Netscape. We don't have any control over ActiveState's packaging; you'll need to refer to their Web site to find who should hear about this problem. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483805&group_id=5470 From noreply@sourceforge.net Tue Nov 20 15:07:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 07:07:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-480188 ] printing unicode objects Message-ID: Bugs item #480188, was opened at 2001-11-09 13:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: M.-A. Lemburg (lemburg) Summary: printing unicode objects Initial Comment: """ In a print statement using a trailing comma (to suppress printing a newline) printing a unicode object that contains a trailing newline sometimes prints an extra space. If I have a unicode object, u, that contains a trailing newline, then... print str(u), # behaves as expected print u.encode(), # behaves as expected print u, # the newline isn't recognized # and the next print # inserts a space It appears that the implicit form of encoding the unicode object doesn't quite work right with print. The following code illustrates my problem. My output follows. """ import sys print sys.version s = 'This is a test\n' u = unicode(s) print print s, print s, print s, print print str(u), print str(u), print str(u), print print u.encode(), print u.encode(), print u.encode(), print print u, print u, print u, """ My output: 2.1.1 (#1, Aug 28 2001, 19:51:39) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-85)] This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test This is a test """ ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-20 07:07 Message: Logged In: YES user_id=38388 I'll checkin a fix for this today. ---------------------------------------------------------------------- Comment By: Daniel Popowich (dpopowich) Date: 2001-11-09 14:57 Message: Logged In: YES user_id=372379 When I submitted this but I didn't realize the form was going to eat my formatting. The last few lines of the output has leading spaces. Run the sample code to see what I mean. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=480188&group_id=5470 From noreply@sourceforge.net Tue Nov 20 15:20:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 07:20:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-438164 ] Format operator % fails with unicode Message-ID: Bugs item #438164, was opened at 2001-07-03 00:58 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=438164&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Chema Cortés (chemacortes) Assigned to: M.-A. Lemburg (lemburg) Summary: Format operator % fails with unicode Initial Comment: A dictionary with unicode keys raise an error when it is applied to a string throught the format operator %. Here is the ofending code: >>> "%(año)d" % { u"año":2001 } # KeyError: año This fails because u"año" and "año" cannot be compared: byte strings and Unicode strings are comparable only if the byte string is ASCII. > >>> u"%(año)4d" % { u"año":2001 } # KeyError: año This fails because the key is converted to UTF-8 before lookup. Must be add the unicode case into the format operator % ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-20 07:20 Message: Logged In: YES user_id=38388 Ok, decided to go with the all Unicode solution. Patch already checked in. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-20 06:20 Message: Logged In: YES user_id=38388 The first case cannot be fixed since there's no way to magically know that the lookup string is in Latin-1. The second is slightly different: this was from the very beginning of the Unicode implementation where we used UTF-8 as default encoding. I'll have to think about this some more, but I believe that we should drop the conversion altogether and use the Unicode object for the lookup rather than converting it to a string. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 11:02 Message: Logged In: YES user_id=6380 Wow, that's some vacation! :-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-08-16 03:47 Message: Logged In: YES user_id=38388 I'll look into this after I'm back from vacation on the 10.09. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=438164&group_id=5470 From noreply@sourceforge.net Tue Nov 20 15:21:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 07:21:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-483815 ] "get_referents" is a bad name Message-ID: Bugs item #483815, was opened at 2001-11-20 07:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483815&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Nobody/Anonymous (nobody) Summary: "get_referents" is a bad name Initial Comment: I'm very pleased to see `gc.get_referents()' in 2.2! It is a very helpful function. But `referent' can mean either the referrer or the referree. webster.com defines it as "one that refers or is referred to; especially : the thing that a symbol (as a word or sign) stands for". That is: it allows either, but "especially" the referree. The meaning of `gc.get_referents()' is to get the referrers. I suggest to change the name to `gc.get_referrers()'. Regards, Zooko ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483815&group_id=5470 From noreply@sourceforge.net Tue Nov 20 16:58:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 08:58:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-483815 ] "get_referents" is a bad name Message-ID: Bugs item #483815, was opened at 2001-11-20 07:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483815&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Nobody/Anonymous (nobody) >Summary: "get_referents" is a bad name Initial Comment: I'm very pleased to see `gc.get_referents()' in 2.2! It is a very helpful function. But `referent' can mean either the referrer or the referree. webster.com defines it as "one that refers or is referred to; especially : the thing that a symbol (as a word or sign) stands for". That is: it allows either, but "especially" the referree. The meaning of `gc.get_referents()' is to get the referrers. I suggest to change the name to `gc.get_referrers()'. Regards, Zooko ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-20 08:58 Message: Logged In: YES user_id=6380 I agree with Zooko. Now is a good time to change this! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483815&group_id=5470 From noreply@sourceforge.net Tue Nov 20 16:58:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 08:58:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-483802 ] How to Crash Python In 3 Easy Steps :) Message-ID: Bugs item #483802, was opened at 2001-11-20 06:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483802&group_id=5470 Category: Python Interpreter Core Group: Platform-specific Status: Open >Resolution: Fixed Priority: 5 Submitted By: Andrew Bennetts (spiv) >Assigned to: Anthony Baxter (anthonybaxter) Summary: How to Crash Python In 3 Easy Steps :) Initial Comment: This is easy to demonstrate: Python 2.1.1 (#1, Aug 1 2001, 23:36:34) [GCC 2.95.4 20010319 (Debian prerelease)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import sys >>> del sys.modules['__builtin__'] >>> int('1') Segmentation fault I don't have a copy of any of the 2.2 betas, so I haven't tested it in those. Apologies if this bug has already been fixed. I realise this is a silly thing to even try to do... I was just wondering what would happen. 1.5.2 and 2.0.1 appear to be fine, incidentally. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-20 08:58 Message: Logged In: YES user_id=21627 This is fixed in 2.2, which reports Traceback (most recent call last): File "", line 1, in ? RuntimeError: lost __builtin__ Assigning to Anthony for review as a 2.1.2 bugfix candidate; the patch is in 2.90 of Python/sysmodule.c. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483802&group_id=5470 From noreply@sourceforge.net Tue Nov 20 16:59:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 08:59:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-483653 ] bsddb is not built on RedHat 7.2 Message-ID: Bugs item #483653, was opened at 2001-11-19 18:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bsddb is not built on RedHat 7.2 Initial Comment: Build of bsddb fails under RedHat 7.2 because the appropriate library to link against is libdb-3.2.so which is not checked for by the setup.py script. The fix is trivial and attached along with this message. raj ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-20 08:59 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 From noreply@sourceforge.net Tue Nov 20 17:33:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 09:33:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-483653 ] bsddb is not built on RedHat 7.2 Message-ID: Bugs item #483653, was opened at 2001-11-19 18:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bsddb is not built on RedHat 7.2 Initial Comment: Build of bsddb fails under RedHat 7.2 because the appropriate library to link against is libdb-3.2.so which is not checked for by the setup.py script. The fix is trivial and attached along with this message. raj ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-20 09:33 Message: Logged In: NO Here is the patch to setup.py: --- cut from here ---- *** setup.py Mon Nov 19 21:41:45 2001 --- setup.py.orig Mon Nov 19 21:41:06 2001 *************** *** 390,398 **** # Berkeley DB 3.x.) dblib = [] ! if self.compiler.find_library_file(lib_dirs, 'db-3.2'): ! dblib = ['db-3.2'] ! elif self.compiler.find_library_file(lib_dirs, 'db-3.1'): dblib = ['db-3.1'] elif self.compiler.find_library_file(lib_dirs, 'db3'): dblib = ['db3'] --- 390,396 ---- # Berkeley DB 3.x.) dblib = [] ! if self.compiler.find_library_file(lib_dirs, 'db-3.1'): dblib = ['db-3.1'] elif self.compiler.find_library_file(lib_dirs, 'db3'): dblib = ['db3'] --- cut to here ---- ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-20 08:59 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 From noreply@sourceforge.net Tue Nov 20 17:47:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 09:47:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-483865 ] bsddb on RedHat 7.2 Message-ID: Bugs item #483865, was opened at 2001-11-20 09:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483865&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bsddb on RedHat 7.2 Initial Comment: This is the same as bug #483653. I am trying to make sure that the patch file actually gets attached this time. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483865&group_id=5470 From noreply@sourceforge.net Tue Nov 20 17:50:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 09:50:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-483865 ] bsddb on RedHat 7.2 Message-ID: Bugs item #483865, was opened at 2001-11-20 09:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483865&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bsddb on RedHat 7.2 Initial Comment: This is the same as bug #483653. I am trying to make sure that the patch file actually gets attached this time. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-20 09:50 Message: Logged In: NO Sorry folks, the file doesn't get attached. I checked the check and upload box - honestly. raj ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483865&group_id=5470 From noreply@sourceforge.net Tue Nov 20 20:56:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 12:56:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-483925 ] _hotshot sloppy in error handling Message-ID: Bugs item #483925, was opened at 2001-11-20 12:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483925&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jack Jansen (jackjansen) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: _hotshot sloppy in error handling Initial Comment: _hotshot is rather sloppy in its handling of I/O errors. The low-level flush_data() routine checks for I/O errors, sets an exception and returns -1, but all the routines calling flush_data() then happily ignore this. The effect of this is that the exception is orphaned and will come back to haunt you at a completely unexpected moment. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483925&group_id=5470 From noreply@sourceforge.net Wed Nov 21 03:56:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 20 Nov 2001 19:56:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-483802 ] How to Crash Python In 3 Easy Steps :) Message-ID: Bugs item #483802, was opened at 2001-11-20 06:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483802&group_id=5470 Category: Python Interpreter Core Group: Platform-specific >Status: Closed Resolution: Fixed Priority: 5 Submitted By: Andrew Bennetts (spiv) Assigned to: Anthony Baxter (anthonybaxter) Summary: How to Crash Python In 3 Easy Steps :) Initial Comment: This is easy to demonstrate: Python 2.1.1 (#1, Aug 1 2001, 23:36:34) [GCC 2.95.4 20010319 (Debian prerelease)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import sys >>> del sys.modules['__builtin__'] >>> int('1') Segmentation fault I don't have a copy of any of the 2.2 betas, so I haven't tested it in those. Apologies if this bug has already been fixed. I realise this is a silly thing to even try to do... I was just wondering what would happen. 1.5.2 and 2.0.1 appear to be fine, incidentally. ---------------------------------------------------------------------- >Comment By: Anthony Baxter (anthonybaxter) Date: 2001-11-20 19:56 Message: Logged In: YES user_id=29957 This is committed in version 2.85.2.1 of sysmodule.c Thanks for the headsup. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-20 08:58 Message: Logged In: YES user_id=21627 This is fixed in 2.2, which reports Traceback (most recent call last): File "", line 1, in ? RuntimeError: lost __builtin__ Assigning to Anthony for review as a 2.1.2 bugfix candidate; the patch is in 2.90 of Python/sysmodule.c. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483802&group_id=5470 From noreply@sourceforge.net Wed Nov 21 11:18:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 21 Nov 2001 03:18:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-483653 ] bsddb is not built on RedHat 7.2 Message-ID: Bugs item #483653, was opened at 2001-11-19 18:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 Category: Build Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Martin v. Löwis (loewis) Summary: bsddb is not built on RedHat 7.2 Initial Comment: Build of bsddb fails under RedHat 7.2 because the appropriate library to link against is libdb-3.2.so which is not checked for by the setup.py script. The fix is trivial and attached along with this message. raj ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-20 09:33 Message: Logged In: NO Here is the patch to setup.py: --- cut from here ---- *** setup.py Mon Nov 19 21:41:45 2001 --- setup.py.orig Mon Nov 19 21:41:06 2001 *************** *** 390,398 **** # Berkeley DB 3.x.) dblib = [] ! if self.compiler.find_library_file(lib_dirs, 'db-3.2'): ! dblib = ['db-3.2'] ! elif self.compiler.find_library_file(lib_dirs, 'db-3.1'): dblib = ['db-3.1'] elif self.compiler.find_library_file(lib_dirs, 'db3'): dblib = ['db3'] --- 390,396 ---- # Berkeley DB 3.x.) dblib = [] ! if self.compiler.find_library_file(lib_dirs, 'db-3.1'): dblib = ['db-3.1'] elif self.compiler.find_library_file(lib_dirs, 'db3'): dblib = ['db3'] --- cut to here ---- ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-20 08:59 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 From noreply@sourceforge.net Wed Nov 21 13:01:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 21 Nov 2001 05:01:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-484156 ] no thread support on HP-UX 10.20 Message-ID: Bugs item #484156, was opened at 2001-11-21 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484156&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: no thread support on HP-UX 10.20 Initial Comment: cannot compile with thread support on HP-UX 10.20 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484156&group_id=5470 From noreply@sourceforge.net Wed Nov 21 15:36:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 21 Nov 2001 07:36:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-484208 ] configure, aix (lovely python) Message-ID: Bugs item #484208, was opened at 2001-11-21 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484208&group_id=5470 Category: Installation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Oliver Pitzeier (opitzeier) Assigned to: Nobody/Anonymous (nobody) Summary: configure, aix (lovely python) Initial Comment: i'm so sorry to say this... P Y T H O N is F***IN' BUGGY! ... i'll use perl forever ... i never saw a programming language buggier than this. configure-script is wrong at the line with the ac_ext=C this tippo... (line-nr.: 800) oh my god... and i searched hours and hours... if you don't change it this will happen: -------------------------------------------------- bash-2.04# ./configure creating cache ./config.cache checking MACHDEP... aix5 checking for --without-gcc... checking for --with-cxx=... no checking for c++... no checking for g++... no checking for gcc... gcc checking whether the C++ compiler (gcc ) works... no configure: error: installation or configuration problem: C++ compiler cannot create executables. -------------------------------------------------- due to google and dejanews this is also true for serveral other platforms. have you tested it??? i don't guess so... i used aix 5.1 and python 2.1.1, 2.2b2 (other problems as well!!!) eat your sh** and stop programming! thx! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484208&group_id=5470 From noreply@sourceforge.net Wed Nov 21 16:29:25 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 21 Nov 2001 08:29:25 -0800 Subject: [Python-bugs-list] [ python-Bugs-484208 ] configure, aix (lovely python) Message-ID: Bugs item #484208, was opened at 2001-11-21 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484208&group_id=5470 Category: Installation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Oliver Pitzeier (opitzeier) Assigned to: Nobody/Anonymous (nobody) Summary: configure, aix (lovely python) Initial Comment: i'm so sorry to say this... P Y T H O N is F***IN' BUGGY! ... i'll use perl forever ... i never saw a programming language buggier than this. configure-script is wrong at the line with the ac_ext=C this tippo... (line-nr.: 800) oh my god... and i searched hours and hours... if you don't change it this will happen: -------------------------------------------------- bash-2.04# ./configure creating cache ./config.cache checking MACHDEP... aix5 checking for --without-gcc... checking for --with-cxx=... no checking for c++... no checking for g++... no checking for gcc... gcc checking whether the C++ compiler (gcc ) works... no configure: error: installation or configuration problem: C++ compiler cannot create executables. -------------------------------------------------- due to google and dejanews this is also true for serveral other platforms. have you tested it??? i don't guess so... i used aix 5.1 and python 2.1.1, 2.2b2 (other problems as well!!!) eat your sh** and stop programming! thx! ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-11-21 08:29 Message: Logged In: YES user_id=45365 I'll refrain from answering in your style of language, I'll leave that to someone else. Any takers? :-) But: if you think there is a bug here wrt Python and AIX: please explain what you think the bug is!!! "configure-script is wrong at the line with the ac_ext=C this tippo..." does not constitute a bug report! If I should read "typo" for "tippo": rest assured, the capital C is not a typo. At this point we're testing how to compile C++, not C. And .C files on unix are C++. And if your gcc doesn't compile C++ files you should use --with-cxx. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484208&group_id=5470 From noreply@sourceforge.net Thu Nov 22 02:14:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 21 Nov 2001 18:14:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-410620 ] pydoc installation w.r.t. RPM. Message-ID: Bugs item #410620, was opened at 2001-03-22 15:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=410620&group_id=5470 Category: Installation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Ka-Ping Yee (ping) Summary: pydoc installation w.r.t. RPM. Initial Comment: Overview: When building an RPM the value for the --prefix configure variable takes on double meaning. Why? Well --prefix tells the build system to "build python so that if runs from --prefix at run time". This infers that during "make install" you want to install python in the --prefix area. The problem is when building an RPM you need to build python for --prefix but install into a temprory place (not indicated by the --prefix variable). This is done by overriding the environment variable $prefix during "make install" so rpm can install into a temprary location in order to snarf up the files for packaging. Python in general plays this game well however from 2.1a2 to 2.1b1 the pydoc stuff breaks this paradigm. I'm using this patch in my 2.1b1 rpm specfile but I would prefer the patch to be incorporated into the base installation. -res Patch included below: --------------------- snip ------------------- *** Python-2.1b1/Makefile.pre.in.ORIG Wed Mar 21 13:24:09 2001 --- Python-2.1b1/Makefile.pre.in Wed Mar 21 13:24:48 2001 *************** *** 694,700 **** # This goes into $(exec_prefix) sharedinstall: PYTHONPATH= ./$(PYTHON) $(srcdir)/setup.py install \ ! --install-platlib=$(DESTSHARED) # Build the toplevel Makefile Makefile.pre: Makefile.pre.in config.status --- 694,700 ---- # This goes into $(exec_prefix) sharedinstall: PYTHONPATH= ./$(PYTHON) $(srcdir)/setup.py install \ ! --install-platlib=$(DESTSHARED) --prefix=${prefix} # Build the toplevel Makefile Makefile.pre: Makefile.pre.in config.status ---------------------------------------------------------------------- >Comment By: Ka-Ping Yee (ping) Date: 2001-11-21 18:14 Message: Logged In: YES user_id=45338 This got fixed by patch #471894. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-16 07:25 Message: Logged In: NO I run into the same problem when trying to build/install pyhton-2.1.1 without write-access to /usr/local/bin/ using prefix=... it took quite some time util I understood the problem (it's my first try with python) and found the same fix. why isn't this _correct_ patch included ??? ---------------------------------------------------------------------- Comment By: Anthon van der Neut (anthon) Date: 2001-04-30 04:14 Message: Logged In: YES user_id=90778 I ran into the same problem with 2.1 final release. Tried to use the --root option in that place only to find that the .so files installed by sharedinstall: where now in $(RPM_BUILD_ROOT)/$(RPM_BUILD_ROOT)/.... :( I chose the more restricted patch: --- Makefile.pre.in.org +++ Makefile.pre.in Mon Apr 30 11:33:53 2001 @@ -704,7 +708,7 @@ # This goes into $(exec_prefix) sharedinstall: PYTHONPATH= ./$(PYTHON) $(srcdir)/setup.py install \ - --install-platlib=$(DESTSHARED) + --install-platlib=$(DESTSHARED) --install- scripts=$(BINDIR) # Build the toplevel Makefile Makefile.pre: Makefile.pre.in config.status ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=410620&group_id=5470 From noreply@sourceforge.net Thu Nov 22 03:38:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 21 Nov 2001 19:38:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-455020 ] identify data objects defined in classes Message-ID: Bugs item #455020, was opened at 2001-08-24 10:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=455020&group_id=5470 Category: Python Library Group: Feature Request >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Ka-Ping Yee (ping) Summary: identify data objects defined in classes Initial Comment: If I define a data object at the module level, pydoc identifies it and its value. It would be nice if it did the same for data objects defined in the class scope, e.g.: Class Frame: NO_SHADOW = gtk.SHADOW_NONE SHADOW_IN = gtk.SHADOW_IN ... Skip ---------------------------------------------------------------------- >Comment By: Ka-Ping Yee (ping) Date: 2001-11-21 19:38 Message: Logged In: YES user_id=45338 Tim Peters has done a lot of work on pydoc and has addressed this pretty well in a recent version. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=455020&group_id=5470 From noreply@sourceforge.net Thu Nov 22 03:42:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 21 Nov 2001 19:42:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-427060 ] pydoc: inconsistent behavior for imports Message-ID: Bugs item #427060, was opened at 2001-05-24 13:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=427060&group_id=5470 Category: Python Library Group: None >Status: Pending >Resolution: Later Priority: 5 Submitted By: Bryan Nollett (nollett) Assigned to: Ka-Ping Yee (ping) Summary: pydoc: inconsistent behavior for imports Initial Comment: Consider a module mymod.py containing the following two import statements: from md5 import new from base64 import decode Running "pydoc mymod" yields a helpfile containing a description of function new from md5 but not function decode from base64. The distinction seems to be that new is a built-in function while decode is a "plain" function. From a user's perspective however, this seems to be very inconsistent. >From a very brief glance at pydoc.py, this distinction appears to be an intentional one, but it is not clear to me why this behavior would be desirable, at least in this (very typical?) example. As a further example, consider mymod.py containing a different import statement: from Numeric import array, arange, resize, zeros Some functions in Numeric are built-ins and others "plain" functions. So here "pydoc mymod" would include documentation for array and zeros (since they happen to be implemented as built-ins) and not arange or resize. I think pydoc follows a rule something like this: imported classes -- not documented imported functions -- documented only if built-in imported "data" -- documented (with obvious exceptions) Perhaps the default behavior should be that *none* of the imported symbols be documented? (If all are documented, this makes pydoc much less usable with Numeric or Tkinter where the "from Tkinter import *" idiom is often a necessary evil.) I can see cases where one would like imported symbols to be documented but do not see a mechanism currently in place to distinguish between the two uses. ---------------------------------------------------------------------- >Comment By: Ka-Ping Yee (ping) Date: 2001-11-21 19:42 Message: Logged In: YES user_id=45338 The intent of pydoc was, exactly as you suggest, not to document any of the symbols imported from other modules. The current implementation does the best that it can to do so. The origin of a class can be determined from its __module__ attribute, but unfortunately data objects and built-in functions do not identify the module they come from, so there's no way to tell. A cleverer pydoc might deduce the origin of a symbol from the existence of an import statement in the code, but then i was afraid heuristics might get too tangled for that to be worth it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=427060&group_id=5470 From noreply@sourceforge.net Thu Nov 22 12:43:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 22 Nov 2001 04:43:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-484545 ] win32.odbc failure - description not set Message-ID: Bugs item #484545, was opened at 2001-11-22 04:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484545&group_id=5470 Category: Windows Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: win32.odbc failure - description not set Initial Comment: # _grief.py # This little program crashes Windows 95 at line 12. # It should expire more gracefully. import odbc, os, sys dsn= 'Stocks//' # Works OK, note that null UID and password must be # supplied for Paradox try: print 'dsn= ', dsn s = odbc.odbc(dsn) # dsn is defined in BDE Configurator cur = s.cursor() print cur.description except: print 'grief:', sys.exc_info() print s, cur print 'done' ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484545&group_id=5470 From noreply@sourceforge.net Thu Nov 22 15:49:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 22 Nov 2001 07:49:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-484545 ] win32.odbc failure - description not set Message-ID: Bugs item #484545, was opened at 2001-11-22 04:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484545&group_id=5470 Category: Windows Group: Python 2.2 >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: M.-A. Lemburg (lemburg) Summary: win32.odbc failure - description not set Initial Comment: # _grief.py # This little program crashes Windows 95 at line 12. # It should expire more gracefully. import odbc, os, sys dsn= 'Stocks//' # Works OK, note that null UID and password must be # supplied for Paradox try: print 'dsn= ', dsn s = odbc.odbc(dsn) # dsn is defined in BDE Configurator cur = s.cursor() print cur.description except: print 'grief:', sys.exc_info() print s, cur print 'done' ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-22 07:49 Message: Logged In: YES user_id=38388 Two things: 1. win32 is not supported by the Python-Dev team; it's an ActiveState extension, so support requests should go there. 2. cur.description is only available *after* you have executed at least one statement on the cursor. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484545&group_id=5470 From noreply@sourceforge.net Thu Nov 22 16:46:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 22 Nov 2001 08:46:38 -0800 Subject: [Python-bugs-list] [ python-Bugs-484603 ] SAX Attribute/AttributesNS class missing Message-ID: Bugs item #484603, was opened at 2001-11-22 08:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484603&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Alexandre Fayolle (afayolle) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: SAX Attribute/AttributesNS class missing Initial Comment: In the SAX ContentHandler documentation (html-doc/lib/content-handler-objects.html), references are made to classes that are not described elsewhere, Attributes and AttributesNS. Alexandre ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484603&group_id=5470 From noreply@sourceforge.net Thu Nov 22 17:16:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 22 Nov 2001 09:16:28 -0800 Subject: [Python-bugs-list] [ python-Bugs-484611 ] Typo: spawmv in Modules/posixmodule.c Message-ID: Bugs item #484611, was opened at 2001-11-22 09:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484611&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael Krause (mkrause) Assigned to: Nobody/Anonymous (nobody) Summary: Typo: spawmv in Modules/posixmodule.c Initial Comment: This should be spawnv not spawmv: PyErr_SetString(PyExc_TypeError, "spawmv() arg 2 must be a tuple or list"); Typo in Python-2.2b2, 2.1.1 and current CVS tree. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484611&group_id=5470 From noreply@sourceforge.net Thu Nov 22 19:57:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 22 Nov 2001 11:57:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-484642 ] hotshot not being installed Message-ID: Bugs item #484642, was opened at 2001-11-22 11:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484642&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: hotshot not being installed Initial Comment: hotshot should probably be inserted in LIBSUBDIRS of file Makefile.pre.in since it's not being installed right now. _hotshot is installed as expected though. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484642&group_id=5470 From noreply@sourceforge.net Thu Nov 22 20:07:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 22 Nov 2001 12:07:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-484645 ] little bug in pycodegen.py Message-ID: Bugs item #484645, was opened at 2001-11-22 12:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484645&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: little bug in pycodegen.py Initial Comment: When the script is called directly (__name__ == "__main__"), compileFile() should be called, not compile() as it is now. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484645&group_id=5470 From noreply@sourceforge.net Fri Nov 23 01:38:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 22 Nov 2001 17:38:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-484715 ] Upgrade TCL for windows installer Message-ID: Bugs item #484715, was opened at 2001-11-22 17:38 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484715&group_id=5470 Category: Installation Group: Not a Bug Status: Open Resolution: None Priority: 5 Submitted By: Kirill Simonov (kirill_simonov) Assigned to: Nobody/Anonymous (nobody) Summary: Upgrade TCL for windows installer Initial Comment: Windows installer comes with TCL/TK 8.3.2. The latest version of TCL/TK is 8.3.4. One of the "greatest" features of the latest TCL/TK is the ability to change window icons. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484715&group_id=5470 From noreply@sourceforge.net Fri Nov 23 13:33:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 05:33:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-484857 ] print with comma ended Message-ID: Bugs item #484857, was opened at 2001-11-23 05:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484857&group_id=5470 Category: IDLE Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Hernan Martinez Foffani (hfoffani) Assigned to: Guido van Rossum (gvanrossum) Summary: print with comma ended Initial Comment: python 2.2a1 on windows 2000 professional, and the included IDLE. the following function prints 'AB' in the command line, but 'A B' in IDLE. should print 'AB' as stated in chapter 6.6 item 3 of the Reference manual def test(): print "%c" % 65, sys.stdout.write("") print "%c" % 66 test() ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484857&group_id=5470 From noreply@sourceforge.net Fri Nov 23 13:55:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 05:55:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-484866 ] print '\b', '\a', etc. Message-ID: Bugs item #484866, was opened at 2001-11-23 05:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484866&group_id=5470 Category: IDLE Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Hernan Martinez Foffani (hfoffani) Assigned to: Guido van Rossum (gvanrossum) Summary: print '\b', '\a', etc. Initial Comment: they doesn't "work" in IDLE. (don't know if this IS a bug and if it is if belongs to IDLE or Tk.) python 2.2a1 on windows 2000 professional and the included IDLE ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484866&group_id=5470 From noreply@sourceforge.net Fri Nov 23 18:50:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 10:50:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-484950 ] docs suggest no cyclic garbage collectio Message-ID: Bugs item #484950, was opened at 2001-11-23 10:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: docs suggest no cyclic garbage collectio Initial Comment: http://python.sourceforge.net/devel-docs/ext/refcounts.html Section 1.10 is a bit out of date -- it would lead some readers to believe that Python doesn't collect cyclical garbage. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 From noreply@sourceforge.net Fri Nov 23 19:43:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 11:43:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-484967 ] bad links at Ref Guide Message-ID: Bugs item #484967, was opened at 2001-11-23 11:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484967&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: bad links at Ref Guide Initial Comment: There are broken links and bad formed urls at a couple of html files in the Ref manual. On file: Doc/ref/atom-literals.html node20.html#tok-stringliteral node23.html#tok-longinteger On file: Doc/ref/integers.html <> Are the same problems as the #217195 bug report? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484967&group_id=5470 From noreply@sourceforge.net Fri Nov 23 19:47:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 11:47:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-484967 ] bad links at Ref Guide Message-ID: Bugs item #484967, was opened at 2001-11-23 11:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484967&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: bad links at Ref Guide Initial Comment: There are broken links and bad formed urls at a couple of html files in the Ref manual. On file: Doc/ref/atom-literals.html node20.html#tok-stringliteral node23.html#tok-longinteger On file: Doc/ref/integers.html <> Are the same problems as the #217195 bug report? ---------------------------------------------------------------------- Comment By: Hernan Martinez Foffani (hfoffani) Date: 2001-11-23 11:47 Message: Logged In: YES user_id=112690 "nobody/anonymous" was me: Hernan Foffani. :-) and the errors are in python 2.2b2 docs (windows distrib) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484967&group_id=5470 From noreply@sourceforge.net Fri Nov 23 20:27:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 12:27:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-484977 ] memory leaks with new style classes Message-ID: Bugs item #484977, was opened at 2001-11-23 12:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Thomas Heller (theller) Assigned to: Nobody/Anonymous (nobody) Summary: memory leaks with new style classes Initial Comment: This code leaks 19 references: class X(object): pass This code leaks 12 references: class X(type): pass ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 From noreply@sourceforge.net Fri Nov 23 20:30:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 12:30:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-484977 ] memory leaks with new style classes Message-ID: Bugs item #484977, was opened at 2001-11-23 12:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Thomas Heller (theller) Assigned to: Nobody/Anonymous (nobody) Summary: memory leaks with new style classes Initial Comment: This code leaks 19 references: class X(object): pass This code leaks 12 references: class X(type): pass ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2001-11-23 12:30 Message: Logged In: YES user_id=11105 I forgot: 2.2b2, Windows 2000, debug build. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 From noreply@sourceforge.net Fri Nov 23 22:04:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 14:04:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-482752 ] __getstate__ & __setstate__ ignored Message-ID: Bugs item #482752, was opened at 2001-11-16 22:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None >Priority: 7 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __getstate__ & __setstate__ ignored Initial Comment: pickle does not recognize overloaded __getstate__ or __setstate__ methods on subclasses of python built-in types. Instead, the instances' __dict__ attribute is always pickled. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 From noreply@sourceforge.net Fri Nov 23 22:06:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 14:06:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-484866 ] print '\b', '\a', etc. Message-ID: Bugs item #484866, was opened at 2001-11-23 05:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484866&group_id=5470 Category: IDLE Group: Python 2.2 >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Hernan Martinez Foffani (hfoffani) Assigned to: Guido van Rossum (gvanrossum) Summary: print '\b', '\a', etc. Initial Comment: they doesn't "work" in IDLE. (don't know if this IS a bug and if it is if belongs to IDLE or Tk.) python 2.2a1 on windows 2000 professional and the included IDLE ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 14:06 Message: Logged In: YES user_id=6380 Depends on what you call "don't work". It writes the characters to stdout, and that's all that Python guarantees. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484866&group_id=5470 From noreply@sourceforge.net Fri Nov 23 22:12:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 14:12:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-484208 ] configure, aix (lovely python) Message-ID: Bugs item #484208, was opened at 2001-11-21 07:36 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484208&group_id=5470 Category: Installation Group: None >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: Oliver Pitzeier (opitzeier) Assigned to: Nobody/Anonymous (nobody) Summary: configure, aix (lovely python) Initial Comment: i'm so sorry to say this... P Y T H O N is F***IN' BUGGY! ... i'll use perl forever ... i never saw a programming language buggier than this. configure-script is wrong at the line with the ac_ext=C this tippo... (line-nr.: 800) oh my god... and i searched hours and hours... if you don't change it this will happen: -------------------------------------------------- bash-2.04# ./configure creating cache ./config.cache checking MACHDEP... aix5 checking for --without-gcc... checking for --with-cxx=... no checking for c++... no checking for g++... no checking for gcc... gcc checking whether the C++ compiler (gcc ) works... no configure: error: installation or configuration problem: C++ compiler cannot create executables. -------------------------------------------------- due to google and dejanews this is also true for serveral other platforms. have you tested it??? i don't guess so... i used aix 5.1 and python 2.1.1, 2.2b2 (other problems as well!!!) eat your sh** and stop programming! thx! ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-23 14:12 Message: Logged In: YES user_id=21627 This is not a bug. As Jack indicates, --without-cxx is required if your C++ compiler does not work. If you still think there is a bug in Python (i.e. if your C++ compiler does work, but Python fails to detect that), please submit another bug report. ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-21 08:29 Message: Logged In: YES user_id=45365 I'll refrain from answering in your style of language, I'll leave that to someone else. Any takers? :-) But: if you think there is a bug here wrt Python and AIX: please explain what you think the bug is!!! "configure-script is wrong at the line with the ac_ext=C this tippo..." does not constitute a bug report! If I should read "typo" for "tippo": rest assured, the capital C is not a typo. At this point we're testing how to compile C++, not C. And .C files on unix are C++. And if your gcc doesn't compile C++ files you should use --with-cxx. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484208&group_id=5470 From noreply@sourceforge.net Fri Nov 23 22:14:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 14:14:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-484156 ] no thread support on HP-UX 10.20 Message-ID: Bugs item #484156, was opened at 2001-11-21 05:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484156&group_id=5470 Category: Threads Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: no thread support on HP-UX 10.20 Initial Comment: cannot compile with thread support on HP-UX 10.20 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-23 14:14 Message: Logged In: YES user_id=21627 Can you provide details? Without any details (like: what did you do, what did the computer say, why do you think this is incorrect) we won't be able to do anything about this problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484156&group_id=5470 From noreply@sourceforge.net Fri Nov 23 22:18:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 14:18:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-484994 ] bugs in Tix.py ListNoteBook PanedWindow Message-ID: Bugs item #484994, was opened at 2001-11-23 14:18 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484994&group_id=5470 Category: Tkinter Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Robert Roy (rjroy) Assigned to: Nobody/Anonymous (nobody) Summary: bugs in Tix.py ListNoteBook PanedWindow Initial Comment: Python 2.2b2 Tix.py 1) class ListNoteBook, __init__ method calls base class __init__ method with "tixDirList" instead of "tixListNoteBook" 2) class PanedWindow, panes method, does not split the string returned from self.tk.call(self._w, 'panes') patch also adds: access to the paned window in ListNoteBook class dummyPanedWindow to support above paneconfigure methods to PanedWindow pageConfigure methods to NoteBook and ListNoteBook page and pages methods to ListNoteBook I am attaching the pathched file as well as the following diff ################ 1044c1044,1046 < TixWidget.__init__(self, master, 'tixDirList', ['options'], cnf, kw) --- > TixWidget.__init__(self, master, 'tixListNoteBook', ['options'], cnf, kw) > self.subwidget_list['pane'] = _dummyPanedWindow(self, 'pane', > destroy_physically=0) 1048d1049 < 1054a1056,1070 > def page(self, name): > return self.subwidget(name) > > def pages(self): > # Can't call subwidgets_all directly because we don't want .nbframe > names = self.tk.split(self.tk.call (self._w, 'pages')) > ret = [] > for x in names: > ret.append(self.subwidget(x)) > return ret > > def pageconfigure(self, name, **kw): > apply(self.tk.call, > (self._w, 'pageconfigure', name) + self._options(kw)) > 1099a1116,1119 > def pageconfigure(self, name, **kw): > apply(self.tk.call, > (self._w, 'pageconfigure', name) + self._options(kw)) > 1162c1182 < names = self.tk.call(self._w, 'panes') --- > names = self.tk.split(self.tk.call (self._w, 'panes')) 1167a1188,1191 > def paneconfigure(self, name, **kw): > apply(self.tk.call, > (self._w, 'paneconfigure', name) + self._options(kw)) > 1571a1596,1599 > TixSubWidget.__init__(self, master, name, destroy_physically) > > class _dummyPanedWindow(PanedWindow, TixSubWidget): > def __init__(self, master, name, destroy_physically=1): ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484994&group_id=5470 From noreply@sourceforge.net Sat Nov 24 02:07:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 18:07:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-484857 ] print with comma ended Message-ID: Bugs item #484857, was opened at 2001-11-23 05:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484857&group_id=5470 >Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Hernan Martinez Foffani (hfoffani) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: print with comma ended Initial Comment: python 2.2a1 on windows 2000 professional, and the included IDLE. the following function prints 'AB' in the command line, but 'A B' in IDLE. should print 'AB' as stated in chapter 6.6 item 3 of the Reference manual def test(): print "%c" % 65, sys.stdout.write("") print "%c" % 66 test() ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:07 Message: Logged In: YES user_id=6380 You're right, that's what the docs say. I'm in a predicament though: I found at least half a dozen places in IDLE alone where a class defines a write() method in order to be usable as an output file, and I would have to add "self.softspace = 0" to each of these methods. I'm sure there are tons of other "file-like objects" (e.g. StringIO and cStringIO) that also have this same "deficiency" and would have to be fixed the same way. I'd rather update the docs to clarify that while this is the behavior of the standard file object, not all file-like objects behave this way, and programs shouldn't count on this. If you want a reliable way to get rid of the space, assign zero to sys.stdout.softspace yourself, rather than using sys.stdout.write(""). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484857&group_id=5470 From noreply@sourceforge.net Sat Nov 24 02:08:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 18:08:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-479698 ] __coerce__ not working with new classes Message-ID: Bugs item #479698, was opened at 2001-11-08 10:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479698&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None >Priority: 7 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __coerce__ not working with new classes Initial Comment: I converted a numeric-like class from the "old-style" Python class to the "new-style" Python class by inheriting from "object". Numerical operations like "__mul__" and "__div__" failed with the "new-style" class. After tracking the problem, I found that the "__coerce__" special method was called when performing mixed-type arithmetic, as expected, with the "old-style" class, but not with the "new-style" class. The arithmetic operations failed because the expected type conversion was not performed with the mixed-type arithmetic. As best I can tell, without examining the Python source code, the following is taking place: 1. The documentation for the "__coerce__" special method states that it is invoked in the case of mixed-type arithmetic only if at least one of the arguments in an arithmetic operation is a class instance. 2. An object created with an "old-style" class is of type "instance", as verified by the "type" function, i.e. 3. An object created with a "new-style" class is of a different type, as also verified by the "type function, i.e. Evidently, because of this change in type, the "__coerce__" function is not longer invoked for "new-style" Python classes. Gary H. Loechelt ON Semiconductor ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479698&group_id=5470 From noreply@sourceforge.net Sat Nov 24 02:13:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 18:13:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-484977 ] memory leaks with new style classes Message-ID: Bugs item #484977, was opened at 2001-11-23 12:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 >Category: Type/class unification >Group: Python 2.2 Status: Open >Resolution: Works For Me Priority: 5 Submitted By: Thomas Heller (theller) >Assigned to: Thomas Heller (theller) Summary: memory leaks with new style classes Initial Comment: This code leaks 19 references: class X(object): pass This code leaks 12 references: class X(type): pass ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:13 Message: Logged In: YES user_id=6380 Which version did you try? Which tool did you use? I observed no memory growth for this program (with 2.2b2+): while 1: class C(object): pass class T(type): pass so I conclude there's no leak. There may appear to be a leak, but this is due to circular references and properly taken care of by the garbage collector. So, back to you -- you can either close this, or provide more information and assign it back to me. ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2001-11-23 12:30 Message: Logged In: YES user_id=11105 I forgot: 2.2b2, Windows 2000, debug build. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 From noreply@sourceforge.net Sat Nov 24 02:16:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 18:16:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) >Assigned to: M.-A. Lemburg (lemburg) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-23 18:16 Message: Logged In: YES user_id=31435 It clarifies the problem but mainly in the sense of clarifying that it *is* a problem: we don't require, or even recommend, any particular way of building extensions. IOW, "but I can't be certain" is correct, but not for a reason I can do anything about except to confirm that it's the truth. Marc-Andre Lemburg is our most prolific extension writer, so I'm reassigning this to him in the hope he'll explain what *he* does. You could do a lot worse than catering to whatever that is. ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 15:34 Message: Logged In: YES user_id=52572 The problem goes something like this: Assume I have a nice, generalized build system (I do, but not as nice as it will be soon ;-)), to which I want to add Python support. I want it to be possible to compile/test code against multiple Python versions, though that's not too relevant to the problem. I'd like finding the neccessary components (headers, libs, executables) to be relatively automatic, based on, e.g. the user's specification of a python base directory and a version number. On Unices it should be possible to do without the base directory unless python was configured --with-prefix=.... I have to assume that someone developing Python extension modules may either be using the source tree, or an installation specific to to the platform he's developing on. Someone who's just building an extension module is probably using a Python installation, but I can't be certain. So, does that clear things up a bit? -Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-15 15:15 Message: Logged In: YES user_id=31435 The source distro is just a snapshot of Python's CVS development tree, so it's a build tree, not an installation tree. "The std" Windows installation tree is the one created by the PythonLabs Windows installer. Similar Windows trees are created by the ActiveState and PythonWare Windows installers. "The std" Unix installation tree is whatever the heck "make install" does on Unix (and it's not the *build* tree either). Likewise for Macs, which rearrage the build tree in yet other ways for installation. The relationship between the build tree and the Windows installation tree is established by the 3000-line python20.wse script in PCbuild, which is input to the Wise installer-builder; I can't summarize it usefully here, and Windows installation also involves fiddling with registry entries. Afraid I'm still not clear on why you're asking, although I'm quite sure I'm not really helping . ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Sat Nov 24 02:17:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 18:17:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on unbounded recursion in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 4 Submitted By: Dallas T. Johnston (elaias) Assigned to: Nobody/Anonymous (nobody) Summary: crash on unbounded recursion in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:17 Message: Logged In: YES user_id=6380 Let's be clear about rexec. It is currently a pathetic excuse for security. It *could* be made safe, and this is one of the things that would have to be fixed; but there are so many other places that I don't really care any more whether this particular core dump gets fixed (at least not from a security p.o.v.). It's strange though that even though there's a Python function at each level, the Python stack limit check doesn't trigger before C stack overflows. Maybe it would make more sense to spend some time getting the general stack limit check more robust. Unfortunately this has to be done separately for each C compiler and each operating system variation... :-( ---------------------------------------------------------------------- Comment By: Andrew Bennetts (spiv) Date: 2001-11-20 06:53 Message: Logged In: YES user_id=50945 I agree that segfaults like this are a problem; this is a potential security problem because it means that untrusted code can crash the interpreter, even if you try to use something like rexec. ---------------------------------------------------------------------- Comment By: Dallas T. Johnston (elaias) Date: 2001-11-19 09:45 Message: Logged In: YES user_id=109241 Sorry, I thought that this was obvious enough for the person responsible that I didn't have to explain. Thanks for the correction though. And I would say that a segfault in any program is a problem, especially an interpreter. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Sat Nov 24 02:47:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 18:47:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-483236 ] subclass of int Message-ID: Bugs item #483236, was opened at 2001-11-18 18:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483236&group_id=5470 >Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Guido van Rossum (gvanrossum) Summary: subclass of int Initial Comment: a in the following example should be 220 >>> class ticks(int): ... __slots__ = ['clk'] ... def __init__(self,dd): ... dd*=self.clk ... int.__init__(self,dd) ... >>> ticks.clk=11 >>> a=ticks(20) >>> a 20 ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:47 Message: Logged In: YES user_id=6380 Sorry, you can't reinitialize an int object's value by calling int.__init__() again. The int value initialization happens in int.__new__() and that returns a new object. You could try this: >>> class ticks(int): ... def __new__(cls, dd=0): ... dd*=20 ... return int.__new__(cls, dd) BTW, your use of __slots__ = ['clk'] followed by an assignment to ticks.clk doesn't make sense; the __slots__ magic creates *instance* variables. But that's not related to your complaint. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483236&group_id=5470 From noreply@sourceforge.net Sat Nov 24 04:02:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 20:02:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-482752 ] __getstate__ & __setstate__ ignored Message-ID: Bugs item #482752, was opened at 2001-11-16 22:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __getstate__ & __setstate__ ignored Initial Comment: pickle does not recognize overloaded __getstate__ or __setstate__ methods on subclasses of python built-in types. Instead, the instances' __dict__ attribute is always pickled. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 20:02 Message: Logged In: YES user_id=6380 You're right about __getstate__, but __setstate__ is supported just fine. Please try the patch below and see if that fixes it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 From noreply@sourceforge.net Sat Nov 24 04:02:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 23 Nov 2001 20:02:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-482752 ] __getstate__ & __setstate__ ignored Message-ID: Bugs item #482752, was opened at 2001-11-16 22:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __getstate__ & __setstate__ ignored Initial Comment: pickle does not recognize overloaded __getstate__ or __setstate__ methods on subclasses of python built-in types. Instead, the instances' __dict__ attribute is always pickled. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 20:02 Message: Logged In: YES user_id=6380 You're right about __getstate__, but __setstate__ is supported just fine. Please try the patch below and see if that fixes it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 20:02 Message: Logged In: YES user_id=6380 You're right about __getstate__, but __setstate__ is supported just fine. Please try the patch below and see if that fixes it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 From noreply@sourceforge.net Sat Nov 24 09:26:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 01:26:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-483815 ] "get_referents" is a bad name Message-ID: Bugs item #483815, was opened at 2001-11-20 07:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483815&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Nobody/Anonymous (nobody) >Summary: "get_referents" is a bad name Initial Comment: I'm very pleased to see `gc.get_referents()' in 2.2! It is a very helpful function. But `referent' can mean either the referrer or the referree. webster.com defines it as "one that refers or is referred to; especially : the thing that a symbol (as a word or sign) stands for". That is: it allows either, but "especially" the referree. The meaning of `gc.get_referents()' is to get the referrers. I suggest to change the name to `gc.get_referrers()'. Regards, Zooko ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-24 01:26 Message: Logged In: YES user_id=21627 Fixed in gcmodule.c 2.30; adding documentation in NEWS 1.318 and libgc.tex 1.8. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-20 08:58 Message: Logged In: YES user_id=6380 I agree with Zooko. Now is a good time to change this! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483815&group_id=5470 From noreply@sourceforge.net Sat Nov 24 09:29:21 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 01:29:21 -0800 Subject: [Python-bugs-list] [ python-Bugs-483653 ] bsddb is not built on RedHat 7.2 Message-ID: Bugs item #483653, was opened at 2001-11-19 18:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Martin v. Löwis (loewis) Summary: bsddb is not built on RedHat 7.2 Initial Comment: Build of bsddb fails under RedHat 7.2 because the appropriate library to link against is libdb-3.2.so which is not checked for by the setup.py script. The fix is trivial and attached along with this message. raj ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-24 01:29 Message: Logged In: YES user_id=21627 Tbanks for the patch. Applied as setup.py 1.65. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-20 09:33 Message: Logged In: NO Here is the patch to setup.py: --- cut from here ---- *** setup.py Mon Nov 19 21:41:45 2001 --- setup.py.orig Mon Nov 19 21:41:06 2001 *************** *** 390,398 **** # Berkeley DB 3.x.) dblib = [] ! if self.compiler.find_library_file(lib_dirs, 'db-3.2'): ! dblib = ['db-3.2'] ! elif self.compiler.find_library_file(lib_dirs, 'db-3.1'): dblib = ['db-3.1'] elif self.compiler.find_library_file(lib_dirs, 'db3'): dblib = ['db3'] --- 390,396 ---- # Berkeley DB 3.x.) dblib = [] ! if self.compiler.find_library_file(lib_dirs, 'db-3.1'): dblib = ['db-3.1'] elif self.compiler.find_library_file(lib_dirs, 'db3'): dblib = ['db3'] --- cut to here ---- ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-20 08:59 Message: Logged In: YES user_id=21627 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483653&group_id=5470 From noreply@sourceforge.net Sat Nov 24 09:30:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 01:30:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-483865 ] bsddb on RedHat 7.2 Message-ID: Bugs item #483865, was opened at 2001-11-20 09:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483865&group_id=5470 Category: Build Group: Platform-specific >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bsddb on RedHat 7.2 Initial Comment: This is the same as bug #483653. I am trying to make sure that the patch file actually gets attached this time. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-24 01:30 Message: Logged In: YES user_id=21627 Duplicate of #483653 ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-20 09:50 Message: Logged In: NO Sorry folks, the file doesn't get attached. I checked the check and upload box - honestly. raj ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483865&group_id=5470 From noreply@sourceforge.net Sat Nov 24 09:32:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 01:32:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-484611 ] Typo: spawmv in Modules/posixmodule.c Message-ID: Bugs item #484611, was opened at 2001-11-22 09:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484611&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Michael Krause (mkrause) Assigned to: Nobody/Anonymous (nobody) Summary: Typo: spawmv in Modules/posixmodule.c Initial Comment: This should be spawnv not spawmv: PyErr_SetString(PyExc_TypeError, "spawmv() arg 2 must be a tuple or list"); Typo in Python-2.2b2, 2.1.1 and current CVS tree. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-24 01:32 Message: Logged In: YES user_id=21627 Thanks for the report. Fixed in posixmodule.c 2.208. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484611&group_id=5470 From noreply@sourceforge.net Sat Nov 24 09:39:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 01:39:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-484642 ] hotshot not being installed Message-ID: Bugs item #484642, was opened at 2001-11-22 11:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484642&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: hotshot not being installed Initial Comment: hotshot should probably be inserted in LIBSUBDIRS of file Makefile.pre.in since it's not being installed right now. _hotshot is installed as expected though. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-24 01:39 Message: Logged In: YES user_id=21627 Thanks for the report. Fixed in Makefile.pre.in 1.67. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484642&group_id=5470 From noreply@sourceforge.net Sat Nov 24 10:39:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 02:39:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-485080 ] lib/test/data not installed Message-ID: Bugs item #485080, was opened at 2001-11-24 02:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485080&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Grzegorz Makarewicz (makaron) Assigned to: Nobody/Anonymous (nobody) Summary: lib/test/data not installed Initial Comment: test/data should be inserted in LIBSUBDIRS of file Makefile.pre.in since it's not being installed right now. test/test_email fails. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485080&group_id=5470 From noreply@sourceforge.net Sat Nov 24 18:04:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 10:04:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-484977 ] memory leaks with new style classes Message-ID: Bugs item #484977, was opened at 2001-11-23 12:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: Works For Me Priority: 5 Submitted By: Thomas Heller (theller) Assigned to: Thomas Heller (theller) Summary: memory leaks with new style classes Initial Comment: This code leaks 19 references: class X(object): pass This code leaks 12 references: class X(type): pass ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2001-11-24 10:04 Message: Logged In: YES user_id=33168 I just tried both lines with purify and it reported no leaks on Solaris 8. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:13 Message: Logged In: YES user_id=6380 Which version did you try? Which tool did you use? I observed no memory growth for this program (with 2.2b2+): while 1: class C(object): pass class T(type): pass so I conclude there's no leak. There may appear to be a leak, but this is due to circular references and properly taken care of by the garbage collector. So, back to you -- you can either close this, or provide more information and assign it back to me. ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2001-11-23 12:30 Message: Logged In: YES user_id=11105 I forgot: 2.2b2, Windows 2000, debug build. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 From noreply@sourceforge.net Sat Nov 24 18:08:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 10:08:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-485133 ] memory leak in test_parser Message-ID: Bugs item #485133, was opened at 2001-11-24 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485133&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leak in test_parser Initial Comment: parsermodule.c leaks in recursive calls to build_node_children(). see attached file for info ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485133&group_id=5470 From noreply@sourceforge.net Sat Nov 24 18:45:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 10:45:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-485139 ] mem leak in interpreter from syntax err Message-ID: Bugs item #485139, was opened at 2001-11-24 10:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485139&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: mem leak in interpreter from syntax err Initial Comment: when a syntax error occurs in the interpreter, the interpreter leaks see the attached file for more info ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485139&group_id=5470 From noreply@sourceforge.net Sat Nov 24 18:49:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 10:49:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-485142 ] memory leak in test_binop Message-ID: Bugs item #485142, was opened at 2001-11-24 10:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485142&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leak in test_binop Initial Comment: test_binop leaks memory in longobjects see attached file for detailed info ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485142&group_id=5470 From noreply@sourceforge.net Sat Nov 24 18:54:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 10:54:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-485145 ] memory leak in test_unicode Message-ID: Bugs item #485145, was opened at 2001-11-24 10:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485145&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leak in test_unicode Initial Comment: test_unicode leaks memory see attached file for details ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485145&group_id=5470 From noreply@sourceforge.net Sat Nov 24 19:39:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 11:39:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-485150 ] memory leak from marshal.c (test_email) Message-ID: Bugs item #485150, was opened at 2001-11-24 11:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485150&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leak from marshal.c (test_email) Initial Comment: marshal.c seems to be the culprit in leaking data from running test_email see attached file for details ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485150&group_id=5470 From noreply@sourceforge.net Sat Nov 24 19:45:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 11:45:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-485152 ] memory leak in test_scope Message-ID: Bugs item #485152, was opened at 2001-11-24 11:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485152&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: memory leak in test_scope Initial Comment: test_scope leaks memory see attached file for details ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485152&group_id=5470 From noreply@sourceforge.net Sat Nov 24 19:46:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 11:46:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-485153 ] Erroneous Fail of PyEval_CallObject Message-ID: Bugs item #485153, was opened at 2001-11-24 11:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 Category: None Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Cl. Schmidt (clemm) Assigned to: Nobody/Anonymous (nobody) Summary: Erroneous Fail of PyEval_CallObject Initial Comment: If embedding python v2.1.1 in a Windows MFC project, it shows the following behaviour: When a call to e.g. PyArgParseTuple had failed, a subsequent call to PyEval_CallObject fails. The parameters of the both calls are completely independent and have nothing to do with each other. Trying to retrieve the error text of the failed PyEval_CallObject returns the error of the PyArgParseTuple, namely "new style getargs format but argument list is not a tuple". If the first (erroneous) call to PyArgParseTuple is commented, the PyEval_CallObject works without any problem. The attached code snippet documents the error. No attempts have been made to reproduce the behaviour in other environments. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 From noreply@sourceforge.net Sat Nov 24 20:08:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 12:08:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-485165 ] PyEval_CallObject not in the index Message-ID: Bugs item #485165, was opened at 2001-11-24 12:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485165&group_id=5470 Category: Documentation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Cl. Schmidt (clemm) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: PyEval_CallObject not in the index Initial Comment: In the Python/C API Reference Manual, the function PyEval_CallObject is missing in the index. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485165&group_id=5470 From noreply@sourceforge.net Sat Nov 24 20:37:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 12:37:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-485175 ] buffer overflow in traceback.c Message-ID: Bugs item #485175, was opened at 2001-11-24 12:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485175&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Alex Martelli (aleax) Assigned to: Nobody/Anonymous (nobody) Summary: buffer overflow in traceback.c Initial Comment: Running this script: name = 'prova'*200 fou = open('fatto.py','w') print>>fou, 'def',name,'():\n return 1/0' fou.close() import fatto print 'prima (%d)'%len(name) funz = getattr(fatto, name) try: funz() except: print 'beccato' raise Python exits with a segfault. Cause: buffer overflow in traceback.c line 157, the only sprintf -- FMT (wrongly, I surmise) does not limit the number of characters it tries to write to linebuf from argument name, an unbounded-length string (the co_name). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485175&group_id=5470 From noreply@sourceforge.net Sat Nov 24 21:05:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 13:05:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-482752 ] __getstate__ & __setstate__ ignored Message-ID: Bugs item #482752, was opened at 2001-11-16 22:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __getstate__ & __setstate__ ignored Initial Comment: pickle does not recognize overloaded __getstate__ or __setstate__ methods on subclasses of python built-in types. Instead, the instances' __dict__ attribute is always pickled. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-24 13:05 Message: Logged In: YES user_id=6380 I'm not going to wait for feedback from Anonymous -- I've checked in the fix as copy_reg.py rev. 1.9. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 20:02 Message: Logged In: YES user_id=6380 You're right about __getstate__, but __setstate__ is supported just fine. Please try the patch below and see if that fixes it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 20:02 Message: Logged In: YES user_id=6380 You're right about __getstate__, but __setstate__ is supported just fine. Please try the patch below and see if that fixes it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 From noreply@sourceforge.net Sun Nov 25 01:55:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 17:55:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-482752 ] __getstate__ & __setstate__ ignored Message-ID: Bugs item #482752, was opened at 2001-11-16 22:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Closed Resolution: Fixed Priority: 7 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __getstate__ & __setstate__ ignored Initial Comment: pickle does not recognize overloaded __getstate__ or __setstate__ methods on subclasses of python built-in types. Instead, the instances' __dict__ attribute is always pickled. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-24 17:55 Message: Logged In: NO The patch worked. Thank you Guido! ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-24 13:05 Message: Logged In: YES user_id=6380 I'm not going to wait for feedback from Anonymous -- I've checked in the fix as copy_reg.py rev. 1.9. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 20:02 Message: Logged In: YES user_id=6380 You're right about __getstate__, but __setstate__ is supported just fine. Please try the patch below and see if that fixes it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 20:02 Message: Logged In: YES user_id=6380 You're right about __getstate__, but __setstate__ is supported just fine. Please try the patch below and see if that fixes it. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482752&group_id=5470 From noreply@sourceforge.net Sun Nov 25 03:41:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 19:41:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-485240 ] extra arg 'name' for __get|set|del__ Message-ID: Bugs item #485240, was opened at 2001-11-24 19:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485240&group_id=5470 Category: Type/class unification Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: extra arg 'name' for __get|set|del__ Initial Comment: Hello. I am writing an object oriented database library for Python I call ObjFS. While rewriting it for Python 2.2, I found that the special methods __get__, __set__ & __del__ would definately benefit from having the attribute's name as an extra argument: def __get__(self, container, name, oftype=None): def __set__(self, container, name, value): def __del__(self, container, name): In my case, I am using those attribute names to build database keys (a tuple of attribute names ordered from top most container to the current attribute's name) to distribute database content on a shelve database. Without the extra argument, I must hack my own solution using __getattribute__, __setattr__ & __delattr__ (which still achieves 12000 writes/second on my PIII-800). I wouldn't want Python to change on my accord, but I was wondering what the reason was for excluding the argument's name in __get__, __set__ & __del__. The reason I mention this is if it is useful to me, I'm certain it might be useful to many other people. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485240&group_id=5470 From noreply@sourceforge.net Sun Nov 25 04:24:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 20:24:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-485240 ] extra arg 'name' for __get|set|del__ Message-ID: Bugs item #485240, was opened at 2001-11-24 19:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485240&group_id=5470 Category: Type/class unification Group: Feature Request >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: extra arg 'name' for __get|set|del__ Initial Comment: Hello. I am writing an object oriented database library for Python I call ObjFS. While rewriting it for Python 2.2, I found that the special methods __get__, __set__ & __del__ would definately benefit from having the attribute's name as an extra argument: def __get__(self, container, name, oftype=None): def __set__(self, container, name, value): def __del__(self, container, name): In my case, I am using those attribute names to build database keys (a tuple of attribute names ordered from top most container to the current attribute's name) to distribute database content on a shelve database. Without the extra argument, I must hack my own solution using __getattribute__, __setattr__ & __delattr__ (which still achieves 12000 writes/second on my PIII-800). I wouldn't want Python to change on my accord, but I was wondering what the reason was for excluding the argument's name in __get__, __set__ & __del__. The reason I mention this is if it is useful to me, I'm certain it might be useful to many other people. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-24 20:24 Message: Logged In: YES user_id=6380 Passing the name would be redundant for the typical use, where each attribute is described by a separate descriptor. I guess you're using it slightly wrong, attempting to use a single descriptor to describe multiple attributes. (BTW, who are you? Please log in! Also, better come up with a more imaginative name than ObjFS. :-) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485240&group_id=5470 From noreply@sourceforge.net Sun Nov 25 05:48:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sat, 24 Nov 2001 21:48:33 -0800 Subject: [Python-bugs-list] [ python-Bugs-485252 ] weird HTML; typo Message-ID: Bugs item #485252, was opened at 2001-11-24 21:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485252&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: weird HTML; typo Initial Comment: On http://python.sourceforge.net/devel-docs/lib/module-pickle.html, I see several strange things. - 'marshaling' should be spelled 'marshalling' (like everywhere else) - the references to the cPickle and marshal module contain double text: first the module name is printed, then it is printed again as a hyperlink. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485252&group_id=5470 From noreply@sourceforge.net Sun Nov 25 17:01:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 25 Nov 2001 09:01:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-485342 ] fcntl.lockf broken? Message-ID: Bugs item #485342, was opened at 2001-11-25 09:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485342&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Nobody/Anonymous (nobody) Summary: fcntl.lockf broken? Initial Comment: fcntl.lockf appears broken to me. It always gives me "bad file descriptor", even though I can use the file object from which it is derived: >>> import fcntl >>> f = open("names.db", "r") >>> fcntl.lockf(f.fileno(), fcntl.LOCK_EX) Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 9] Bad file descriptor >>> data = f.read() >>> len(data) 45056 Since posixfile is officially deprecated, this would be nice to have working... Environment: Mandrake 8.1, gcc 2.96, glibc 2.2.4 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485342&group_id=5470 From noreply@sourceforge.net Sun Nov 25 17:13:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 25 Nov 2001 09:13:33 -0800 Subject: [Python-bugs-list] [ python-Bugs-485342 ] fcntl.lockf broken? Message-ID: Bugs item #485342, was opened at 2001-11-25 09:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485342&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: Skip Montanaro (montanaro) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: fcntl.lockf broken? Initial Comment: fcntl.lockf appears broken to me. It always gives me "bad file descriptor", even though I can use the file object from which it is derived: >>> import fcntl >>> f = open("names.db", "r") >>> fcntl.lockf(f.fileno(), fcntl.LOCK_EX) Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 9] Bad file descriptor >>> data = f.read() >>> len(data) 45056 Since posixfile is officially deprecated, this would be nice to have working... Environment: Mandrake 8.1, gcc 2.96, glibc 2.2.4 ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-11-25 09:13 Message: Logged In: YES user_id=44345 I retract my contention that there is a bug in lockf. It appears that you can't ask for an exclusive lock on a file which is open for read access. As it's not clear from either the fcntl module documentation or the fcntl, lockf or flock man pages (just what does fcntl.lockf emulate by the way?), perhaps something simple should be added to the fcntl.lockf description. Suggestion: On at least some systems, LOCK_EX can only be used if the file descriptor refers to a file opened for writing. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485342&group_id=5470 From noreply@sourceforge.net Mon Nov 26 01:50:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Sun, 25 Nov 2001 17:50:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-485446 ] tutorial: backticks, repr(), str() Message-ID: Bugs item #485446, was opened at 2001-11-25 17:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485446&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: tutorial: backticks, repr(), str() Initial Comment: http://python.sourceforge.net/devel-docs/tut/node9.html: > how do you convert values to strings? Luckily, Python > has a way to convert any value to a string: pass it to > the repr() function, or just write the value between > reverse quotes (``). Some examples: a) str() should probably be mentioned b) is it good to teach newbies to use backticks? i don't remember when i've last seen them in python code and it's easy to confuse them with single quotes. -- erno@iki.fi ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485446&group_id=5470 From noreply@sourceforge.net Mon Nov 26 10:35:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 02:35:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-485554 ] Copy menu not enabled Message-ID: Bugs item #485554, was opened at 2001-11-26 02:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485554&group_id=5470 Category: Macintosh Group: None Status: Open Resolution: None Priority: 7 Submitted By: Jack Jansen (jackjansen) Assigned to: Jack Jansen (jackjansen) Summary: Copy menu not enabled Initial Comment: The "copy" menu command on the console window never seems to be enabled, as of recently (2.2b2). "Paste" is fine. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485554&group_id=5470 From noreply@sourceforge.net Mon Nov 26 10:41:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 02:41:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-485557 ] MacPython doesnt run on 8.1 Message-ID: Bugs item #485557, was opened at 2001-11-26 02:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485557&group_id=5470 Category: Macintosh Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jack Jansen (jackjansen) Assigned to: Jack Jansen (jackjansen) Summary: MacPython doesnt run on 8.1 Initial Comment: MacPython no longer runs on MacOS 8.1. The problem seems to be hard-linking against DialogsLib. Check whether there is an easy way to weak-link against it (the routines in DialogsLib are of minor importance only, and not used by IDE or any other part of the distribution), otherwise 8.1 support will have to be dropped. Adapt the readme and installer minimal system version settings. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485557&group_id=5470 From noreply@sourceforge.net Mon Nov 26 13:27:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 05:27:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: M.-A. Lemburg (lemburg) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-26 05:27 Message: Logged In: YES user_id=38388 Here's a summary of what I do to support the mx Extensions for multiple Python versions: Note: I use distutils on all platforms and distutils has builtin ways to find where the specific Python version I am targetting lives. On Unix: Since the only binary format distutils currently support is RPM, this is only relevant to the bdist_rpm command. Unfortunately, RPM does not inherit the environment, so you have to hard-code the path to the Python version into the RPM. It goes something like this: $(PYTHON) setup.py bdist_rpm \ --python $(PYTHON) \ --release $(PACKAGERELEASE) \ $(_RPM_REQUIRES) On Windows: Here, you have to run the setup.py file using the Python version you wish to build for, e.g. python2.0 setup.py bdist_win. I choose the Python version by hacking the PATH to point to the right version (this is needed to have sub-processes use the same Python version), e.g. wininst20.bat: @echo off call vcvars32 echo -------------------------------------------------------------- echo Building for Python 2.0... set PATH="d:\Python20;%PATH%" python setup.py clean --all python setup.py bdist_wininst -c -o That's about it. Hope it helps. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-23 18:16 Message: Logged In: YES user_id=31435 It clarifies the problem but mainly in the sense of clarifying that it *is* a problem: we don't require, or even recommend, any particular way of building extensions. IOW, "but I can't be certain" is correct, but not for a reason I can do anything about except to confirm that it's the truth. Marc-Andre Lemburg is our most prolific extension writer, so I'm reassigning this to him in the hope he'll explain what *he* does. You could do a lot worse than catering to whatever that is. ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 15:34 Message: Logged In: YES user_id=52572 The problem goes something like this: Assume I have a nice, generalized build system (I do, but not as nice as it will be soon ;-)), to which I want to add Python support. I want it to be possible to compile/test code against multiple Python versions, though that's not too relevant to the problem. I'd like finding the neccessary components (headers, libs, executables) to be relatively automatic, based on, e.g. the user's specification of a python base directory and a version number. On Unices it should be possible to do without the base directory unless python was configured --with-prefix=.... I have to assume that someone developing Python extension modules may either be using the source tree, or an installation specific to to the platform he's developing on. Someone who's just building an extension module is probably using a Python installation, but I can't be certain. So, does that clear things up a bit? -Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-15 15:15 Message: Logged In: YES user_id=31435 The source distro is just a snapshot of Python's CVS development tree, so it's a build tree, not an installation tree. "The std" Windows installation tree is the one created by the PythonLabs Windows installer. Similar Windows trees are created by the ActiveState and PythonWare Windows installers. "The std" Unix installation tree is whatever the heck "make install" does on Unix (and it's not the *build* tree either). Likewise for Macs, which rearrage the build tree in yet other ways for installation. The relationship between the build tree and the Windows installation tree is established by the 3000-line python20.wse script in PCbuild, which is input to the Wise installer-builder; I can't summarize it usefully here, and Windows installation also involves fiddling with registry entries. Afraid I'm still not clear on why you're asking, although I'm quite sure I'm not really helping . ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Mon Nov 26 14:01:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 06:01:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-485602 ] new module broken with new classes Message-ID: Bugs item #485602, was opened at 2001-11-26 06:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Nobody/Anonymous (nobody) Summary: new module broken with new classes Initial Comment: The new module doesn't support the new-style classes. Example: import new class foo(object): pass a = new.instance(foo,{}) I didn't really expect this to work, but I'm trying to figure out how to emulate the behavior of this function with new-style classes (i.e., how to bring a new-style instance into being without calling __init__). Does a call to PyType_GenericAlloc(PyTypeObject *type, 1) emulate this behavior? Please advise. Cheers, Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 From noreply@sourceforge.net Mon Nov 26 15:59:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 07:59:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-479698 ] __coerce__ not working with new classes Message-ID: Bugs item #479698, was opened at 2001-11-08 10:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479698&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: __coerce__ not working with new classes Initial Comment: I converted a numeric-like class from the "old-style" Python class to the "new-style" Python class by inheriting from "object". Numerical operations like "__mul__" and "__div__" failed with the "new-style" class. After tracking the problem, I found that the "__coerce__" special method was called when performing mixed-type arithmetic, as expected, with the "old-style" class, but not with the "new-style" class. The arithmetic operations failed because the expected type conversion was not performed with the mixed-type arithmetic. As best I can tell, without examining the Python source code, the following is taking place: 1. The documentation for the "__coerce__" special method states that it is invoked in the case of mixed-type arithmetic only if at least one of the arguments in an arithmetic operation is a class instance. 2. An object created with an "old-style" class is of type "instance", as verified by the "type" function, i.e. 3. An object created with a "new-style" class is of a different type, as also verified by the "type function, i.e. Evidently, because of this change in type, the "__coerce__" function is not longer invoked for "new-style" Python classes. Gary H. Loechelt ON Semiconductor ---------------------------------------------------------------------- Comment By: Gary H. Loechelt (loechelt) Date: 2001-11-26 07:59 Message: Logged In: YES user_id=142817 After further looking into this, I am not sure that it is a bug "per se". It may have been the intentionally designed behavior. It is certainly possible for a user to rewrite arithmetic methods (i.e. '__add__') to explicitly call a coerce method. However, in the very least, changing from the "old-style" to the "new-style" class does break existing code that relies upon the '__coerce__' method, and this fact is not very well documented. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479698&group_id=5470 From noreply@sourceforge.net Mon Nov 26 16:56:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 08:56:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-485678 ] Use of __del__ on descriptors ambiguous Message-ID: Bugs item #485678, was opened at 2001-11-26 08:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485678&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Use of __del__ on descriptors ambiguous Initial Comment: When deleting an attribute described by a descriptor implemented in Python, the descriptor's __del__ method is called by the slot_tp_descr_set dispatch function. This is bogus -- __del__ already has a different meaning. This should be renamed to __delete__. (XXX Should we also rename __get__ and __set__ to something more descriptive, e.g. __descr_get__ and and __descr_set__? Or maybe we should use __prop_ as a prefix?) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485678&group_id=5470 From noreply@sourceforge.net Mon Nov 26 16:56:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 08:56:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-485679 ] 2.2b2 Won't build on Solaris 2.8 Message-ID: Bugs item #485679, was opened at 2001-11-26 08:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485679&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Nobody/Anonymous (nobody) Summary: 2.2b2 Won't build on Solaris 2.8 Initial Comment: 2.2b2 doesn't build on Sparc Solaris 2.8 with gcc-2.95.2 and gmake-3.77. The following error occurs: make: *** No rule to make target `Python/thread_*.h', needed by `Python/thread.o'. Stop. This occurs with the standard build process (./configure; make). If I had to take a wild guess on this, perhaps the build process (configure or make) is using a non-portable feature of the shell and/or make. -- Dave ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485679&group_id=5470 From noreply@sourceforge.net Mon Nov 26 18:47:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 10:47:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-485723 ] time.localtime() unsliceable!! Message-ID: Bugs item #485723, was opened at 2001-11-26 10:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485723&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: time.localtime() unsliceable!! Initial Comment: The result of time.localtime()[2:4] (or any slice) will return a strange result, to wit: (, ) Comparison of this slice to any other value will cause Python to CORE. For example: if time.localtime()[3:5] > (12, 00): will CORE python. This is version 2.2b1 on Linux. The workaround I have used is to make a shallow copy of the result of time.localtime(). That is, time.localtime()[:][2:4] works fine! Also, single element indexing DOES seem to work. It is just slicing that causes the problem. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485723&group_id=5470 From noreply@sourceforge.net Mon Nov 26 20:33:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 12:33:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-485766 ] install hangs Message-ID: Bugs item #485766, was opened at 2001-11-26 12:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485766&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: install hangs Initial Comment: Install on NT 4.0 appears to go fine until the final step. It says it is finished, but clicking on OK (or finish) leaves server with a blue screen (python install, not system crash). System just sits there & does not change.. CPU at 30% or less...mostly less. Help! User is frantic! Thanks! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485766&group_id=5470 From noreply@sourceforge.net Mon Nov 26 21:16:54 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 13:16:54 -0800 Subject: [Python-bugs-list] [ python-Bugs-485781 ] get_refer{ent,rer}s() doesn't use `is' Message-ID: Bugs item #485781, was opened at 2001-11-26 13:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Nobody/Anonymous (nobody) Summary: get_refer{ent,rer}s() doesn't use `is' Initial Comment: in Python 2.2b2 as packaged by debian, gc.get_referents() seems to be using equality testing rather than identity testing to find referents. Example: >>> l = [0,3,2,{},] >>> gc.get_referents({}) [Type help() for interactive help, or help(object) for help about object., [0, 3, 2, {}]] The list should not have appeared in the result. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 From noreply@sourceforge.net Mon Nov 26 21:31:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 13:31:12 -0800 Subject: [Python-bugs-list] [ python-Bugs-485252 ] weird HTML; typo Message-ID: Bugs item #485252, was opened at 2001-11-24 21:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485252&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: weird HTML; typo Initial Comment: On http://python.sourceforge.net/devel-docs/lib/module-pickle.html, I see several strange things. - 'marshaling' should be spelled 'marshalling' (like everywhere else) - the references to the cPickle and marshal module contain double text: first the module name is printed, then it is printed again as a hyperlink. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-26 13:31 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/libpickle.tex revision 1.33. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485252&group_id=5470 From noreply@sourceforge.net Mon Nov 26 21:51:04 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 13:51:04 -0800 Subject: [Python-bugs-list] [ python-Bugs-458447 ] New httplib lacks documentation Message-ID: Bugs item #458447, was opened at 2001-09-04 10:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458447&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None >Priority: 6 Submitted By: Rich Salz (rsalz) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: New httplib lacks documentation Initial Comment: urllib should allow the user to set the content-type. This would allow applications to do more than just "forms posting" -- e.g., use it to make SOAP requests. The attached patch allows that. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-26 13:51 Message: Logged In: YES user_id=3066 I have contributed docs for the new version waiting for review in my inbox; these should be checked in this week. Bumping priority so this stays visible to me. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 08:37 Message: Logged In: YES user_id=6380 Fred: it turns out the httplib docs still document the old version of the module. The new version is considerably more powerful, and is essentially undocumented. Maybe you can shame Greg Stein into providing some docs, or maybe you can convert the copious docstrings to LaTeX. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-09-04 20:45 Message: Logged In: YES user_id=44345 I'm not sure why you'd want to program SOAP (or XMLRPC, for that matter) directly. You would (normally) only use those protocols through special-purpose modules like SOAP.py or xmlrpclib.py, both of which talk to httplib I believe. I don't see that allowing the Content-Type: header to be overridden at the urllib level is all that necessary. If you're going to want to mess with Content-Type: you're probably going to want to mess with other headers as well. ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-09-04 19:02 Message: Logged In: YES user_id=36737 I created this bug because urllib didn't let you set the content-type header. Guido pointed out that I should be using httplib. I didn't use httplib because I didn't know until I read the source, that httplib had an https object that used SSL. So I guess "document the https object" is what this has mutated into. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-09-04 14:03 Message: Logged In: YES user_id=3066 It's not at all clear what sort of documentation update is needed. What did you have in mind? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-04 12:20 Message: Logged In: YES user_id=6380 Please don't Delete requests. Please don't Close them unless you are a project manager (I consider it a SF bug that the submitter can decide to close a bug report if they happen to have admin perms on another project). I've reopened this and recategorized as Doc and assigned to Fred so he can see if the httplib docs need to be updated. ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-09-04 12:16 Message: Logged In: YES user_id=36737 Okay, I'll accept that. Had HTTP documented that it included HTTPS I wouldn't have made the initial mistake. :) Let me know if you want the diff. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-04 11:20 Message: Logged In: YES user_id=6380 Isn't SOAP an HTTP application? Then you shouldn't be using urllib, but httplib. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-04 11:20 Message: Logged In: YES user_id=6380 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458447&group_id=5470 From noreply@sourceforge.net Tue Nov 27 00:29:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 16:29:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-485723 ] time.localtime() unsliceable!! Message-ID: Bugs item #485723, was opened at 2001-11-26 10:47 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485723&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: time.localtime() unsliceable!! Initial Comment: The result of time.localtime()[2:4] (or any slice) will return a strange result, to wit: (, ) Comparison of this slice to any other value will cause Python to CORE. For example: if time.localtime()[3:5] > (12, 00): will CORE python. This is version 2.2b1 on Linux. The workaround I have used is to make a shallow copy of the result of time.localtime(). That is, time.localtime()[:][2:4] works fine! Also, single element indexing DOES seem to work. It is just slicing that causes the problem. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-26 16:29 Message: Logged In: YES user_id=31435 Get 2.2b2 (this was fixed some time ago). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485723&group_id=5470 From noreply@sourceforge.net Tue Nov 27 00:33:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 16:33:07 -0800 Subject: [Python-bugs-list] [ python-Bugs-485766 ] install hangs Message-ID: Bugs item #485766, was opened at 2001-11-26 12:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485766&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: install hangs Initial Comment: Install on NT 4.0 appears to go fine until the final step. It says it is finished, but clicking on OK (or finish) leaves server with a blue screen (python install, not system crash). System just sits there & does not change.. CPU at 30% or less...mostly less. Help! User is frantic! Thanks! ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-26 16:33 Message: Logged In: YES user_id=31435 Log in to an Administrator account. Don't install over a network (install from local copy). Accept defaults. Ensure there's enough disk space. Kill virus checkers (etc) for the duration ... this is all "general flaky Windows" advice, there's nothing specific to Python here. If all else fails, try the ActiveState or Secret Labs installer. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485766&group_id=5470 From noreply@sourceforge.net Tue Nov 27 00:44:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 16:44:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-210633 ] urlparse (PR#286) Message-ID: Bugs item #210633, was opened at 2000-07-31 14:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210633&group_id=5470 Category: Python Library Group: Not a Bug Status: Closed Resolution: Invalid Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: urlparse (PR#286) Initial Comment: Jitterbug-Id: 286 Submitted-By: alex@shop.com Date: Mon, 10 Apr 2000 16:40:57 -0400 (EDT) Version: >=1.5 OS: win32 linux urlparse requires that the url contain a "/" so that urlparse("http://foo.com?q=a#blah") results in ("http","foo.com?q=a#blah",....) urlparse should not require slashes in urls that have fragments or query strings. ==================================================================== Audit trail: Tue Jul 11 08:29:15 2000 guido moved from incoming to open ---------------------------------------------------------------------- Comment By: Aaron Swartz (aaronsw) Date: 2001-11-26 16:44 Message: Logged In: YES user_id=122141 RFC2396, not RFC1738 is the latest RFC for URI/URL defintions. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2000-08-24 08:07 Message: RFC 1738, section 3.3, discusses the syntax for HTTP URLs. It implies that the "/" between the is required if either the path of searchpart of the URL is provided, but is not completely clear. I don't see anything relevant in RFC 1945 (HTTP 1.0), but RFC 2616 (HTTP 1.1), section 3.2.2 clearly indicates that the search part should only exist as a part of the path component, which is required to include the leading "/". There is some confusion as to how this should relate to parsing of relative URLs (RFC 1808). This bug can be re-opened if there's evidence urlparse is actually wrong or inconsistent with other URL parsers. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2000-08-16 18:54 Message: Assigned to me so I can deal with urlparse all at once. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=210633&group_id=5470 From noreply@sourceforge.net Tue Nov 27 02:06:11 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 18:06:11 -0800 Subject: [Python-bugs-list] [ python-Bugs-485860 ] Broken links in docs - section 15.2 SHA Message-ID: Bugs item #485860, was opened at 2001-11-26 18:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485860&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Broken links in docs - section 15.2 SHA Initial Comment: Under 'See Also:' in section '15.2 sha -- SHA message digest algorithm' The external links are broken, the new links for the section should be: http://csrc.nist.gov/publications/fips/fips180- 1/fip180-1.txt and http://csrc.nist.gov/publications/fips/fips180- 1/fip180-1.pdf ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485860&group_id=5470 From noreply@sourceforge.net Tue Nov 27 07:22:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Mon, 26 Nov 2001 23:22:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-485781 ] get_refer{ent,rer}s() doesn't use `is' Message-ID: Bugs item #485781, was opened at 2001-11-26 13:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None >Priority: 7 Submitted By: Zooko Ozoko (zooko) >Assigned to: Martin v. Löwis (loewis) Summary: get_refer{ent,rer}s() doesn't use `is' Initial Comment: in Python 2.2b2 as packaged by debian, gc.get_referents() seems to be using equality testing rather than identity testing to find referents. Example: >>> l = [0,3,2,{},] >>> gc.get_referents({}) [Type help() for interactive help, or help(object) for help about object., [0, 3, 2, {}]] The list should not have appeared in the result. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-26 23:22 Message: Logged In: YES user_id=21627 Please try the attached patch (or report back if you cannot). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 From noreply@sourceforge.net Tue Nov 27 08:34:02 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 00:34:02 -0800 Subject: [Python-bugs-list] [ python-Bugs-485860 ] Broken links in docs - section 15.2 SHA Message-ID: Bugs item #485860, was opened at 2001-11-26 18:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485860&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Duplicate Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Broken links in docs - section 15.2 SHA Initial Comment: Under 'See Also:' in section '15.2 sha -- SHA message digest algorithm' The external links are broken, the new links for the section should be: http://csrc.nist.gov/publications/fips/fips180- 1/fip180-1.txt and http://csrc.nist.gov/publications/fips/fips180- 1/fip180-1.pdf ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-27 00:34 Message: Logged In: YES user_id=3066 Duplicate of SF bug #454917. These have already been fixed in the documentation sources; see the development version of the documentation provided as part of the Python 2.2 betas, or http://python.sourceforge.net/devel-docs/lib/modindex.html. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485860&group_id=5470 From noreply@sourceforge.net Tue Nov 27 10:25:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 02:25:12 -0800 Subject: [Python-bugs-list] [ python-Bugs-485951 ] repr diff between string and unicode. Message-ID: Bugs item #485951, was opened at 2001-11-27 02:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 Category: Unicode Group: None Status: Open Resolution: None Priority: 3 Submitted By: Finn Bock (bckfnn) Assigned to: M.-A. Lemburg (lemburg) Summary: repr diff between string and unicode. Initial Comment: A minor difference exists between the repr output of string and unicode. >>> "\x7f" '\x7f' >>> u"\x7f" u'⌂' >>> I can't think of any reason for this difference. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 From noreply@sourceforge.net Tue Nov 27 16:05:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 08:05:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-424002 ] sys.executable not reliable Message-ID: Bugs item #424002, was opened at 2001-05-14 11:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=424002&group_id=5470 Category: Python Library Group: None Status: Closed Resolution: Fixed Priority: 5 Submitted By: Evan Simpson (evansimpson) Assigned to: Guido van Rossum (gvanrossum) Summary: sys.executable not reliable Initial Comment: I created a file test.py containing: #!/usr/local/bin/python2.1 import sys print sys.executable When I run it using the command line "/usr/local/bin/python2.1 test.py", it prints the full path to the executable. When I run it from "/usr/local", using "bin/python2.1 test.py", then I get "bin/python2.1". When I run it using "./test.py", I get a blank string. Under version 1.5.2, all of these yielded the full path to the executable. ---------------------------------------------------------------------- >Comment By: Evan Simpson (evansimpson) Date: 2001-11-27 08:05 Message: Logged In: YES user_id=219362 Yep, it works. Thanks! ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-28 13:00 Message: Logged In: YES user_id=6380 I think I've fixed this in CVS now (post-2.2a4). Can you test this with the new CVS? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 12:06 Message: Logged In: YES user_id=6380 Curious. I'll look into this. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=424002&group_id=5470 From noreply@sourceforge.net Tue Nov 27 17:34:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 09:34:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-486079 ] poplib contains unuseable code Message-ID: Bugs item #486079, was opened at 2001-11-27 09:34 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486079&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: Jon Nelson (jnelson) Assigned to: Nobody/Anonymous (nobody) Summary: poplib contains unuseable code Initial Comment: poplib contains a number of debugging statements, quite useful for debugging, which are turned on via the set_debuglevel function. However, all of the debug statements have been commented out, making it impossible to debug properly. Included is a patch which uncomments the commented code, but does *not* change default behavior. However, if someone issues the set_debuglevel(int N) function, then debugging works, just like the documentation and logic says it should. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486079&group_id=5470 From noreply@sourceforge.net Tue Nov 27 17:37:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 09:37:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-485602 ] new module broken with new classes Message-ID: Bugs item #485602, was opened at 2001-11-26 06:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 >Category: Type/class unification >Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Nobody/Anonymous (nobody) Summary: new module broken with new classes Initial Comment: The new module doesn't support the new-style classes. Example: import new class foo(object): pass a = new.instance(foo,{}) I didn't really expect this to work, but I'm trying to figure out how to emulate the behavior of this function with new-style classes (i.e., how to bring a new-style instance into being without calling __init__). Does a call to PyType_GenericAlloc(PyTypeObject *type, 1) emulate this behavior? Please advise. Cheers, Dave ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 09:37 Message: Logged In: YES user_id=31435 Changed Category & Group. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 From noreply@sourceforge.net Tue Nov 27 17:41:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 09:41:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-481969 ] Doesn't build with OOTB Cygwin Message-ID: Bugs item #481969, was opened at 2001-11-14 21:17 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 Category: Build Group: Python 2.2 >Status: Closed >Resolution: Invalid Priority: 5 Submitted By: David Abrahams (david_abrahams) >Assigned to: Michael Hudson (mwh) Summary: Doesn't build with OOTB Cygwin Initial Comment: It looks like it is getting confused because SIZEOF_LONG doesn't ever get #defined. I made the following patch to pyconfig.h which helped in my case: --- c:/tools/python2.2/include/pyconfig.h Thu Sep 06 01:32:16 2001 +++ c:/tools/Python-2.2b1/Include/pyconfig.h Thu Nov 15 01:25:00 2001 @@ -193,7 +193,7 @@ #endif /* BORLANDC */ /* egcs/gnu-win32 defines __GNUC__ and _WIN32 */ -#if defined(__GNUC__) && defined(_WIN32) +#if defined(__GNUC__) && (defined(_WIN32) || defined (__CYGWIN__)) /* XXX These defines are likely incomplete, but should be easy to fix. They should be complete enough to build extension modules. */ /* Suggested by Rene Liebscher to avoid a GCC 2.91.* @@ -240,7 +240,7 @@ #endif #define HAVE_LONG_LONG 1 -#define LONG_LONG long long +#define LONG_LONG long long #endif /* GNUC */ /* lcc-win32 defines __LCC__ */ ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 09:41 Message: Logged In: YES user_id=31435 Just closed this. ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 08:56 Message: Logged In: YES user_id=52572 Sorry, I just assumed you wouldn't be notified that I still had a question if the bug stayed unassigned. I don't know much about how this tracker is set up. I guess when I tried the technique you propose, the directory was already polluted from the Win32 build. Thanks for all your help, Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 08:50 Message: Logged In: YES user_id=6656 Right, OK. By the time-honoured mkdir build && cd build && ../configure && make trick. By unix-like, I mean unix or cygwin. It seems that doing a VC++ build does something to the tree that prevents such builds working, but as I don't have VC++, I can't say what or what to do about it -- so why did you reassign the bug to me? ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 08:29 Message: Logged In: YES user_id=52572 "Didn't work" means that this #error in PyPort.h fired: #error "LONG_BIT definition appears wrong for platform (bad gcc/glibc config?)." How do you do unix-style builds without polluting the source tree? ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 07:25 Message: Logged In: YES user_id=6656 What does "didn't work" mean? I suspect you've hit the nail on the head as to the cause of your problems, but I know diddly-squat about the VC++ build, so I'm unassigning this from me. I doubt it will be a priority for anyone else, sorry. Note you can do unix-style builds without polluting the source tree (much -- .pycs tend to get left lying around). ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 06:30 Message: Logged In: YES user_id=52572 I have Cygwin 1.3.3-2. The latest appears to be 1.3.5-3, but I got mine just last week. They seem to release something new every few days. I'm using Windows 2000. I've also built and installed gcc-3.0.2 on my Cygwin installation. Oh, maybe this has something to do with it: I previously built a regular Win32 version of Python using MSVC from the same Python source tree. When I first tried to install Python for Cygwin, I made a separate build directory as they recommend for gcc, then did a "../Python-2.2b1/configure ; make" but it didn't work. If indeed the two builds are conflicting, you might consider making this approach possible. The idea is that no configuration-specific cruft gets left in the source tree. This has two advantages: 1. Rebuilds with the same source for different platforms becomes possible 2. You can build from a read-only copy of the sources. -Dave ---------------------------------------------------------------------- Comment By: Michael Hudson (mwh) Date: 2001-11-15 05:10 Message: Logged In: YES user_id=6656 You seem to be editing PC/pyconfig.h, but I don't understand how this file is getting involved in a cygwin build. How are you building Python? It works just fine for me (except for curses, that's a cygwin bug). Also, have you got the latest version of cygwin? What version of windows, etc. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481969&group_id=5470 From noreply@sourceforge.net Tue Nov 27 18:19:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 10:19:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Nobody/Anonymous (nobody) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Tue Nov 27 19:52:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 11:52:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-486144 ] Uninitialized __slot__ vrbl is None Message-ID: Bugs item #486144, was opened at 2001-11-27 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486144&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: Uninitialized __slot__ vrbl is None Initial Comment: >>> class C(object): ... __slots__ = ['a'] ... >>> c = C() >>> print c.a None >>> Nothing else in Python works like this -- a (possibly subclass of) NameError exception would be better. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486144&group_id=5470 From noreply@sourceforge.net Tue Nov 27 19:58:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 11:58:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-484977 ] memory leaks with new style classes Message-ID: Bugs item #484977, was opened at 2001-11-23 12:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed Resolution: Works For Me Priority: 5 Submitted By: Thomas Heller (theller) Assigned to: Thomas Heller (theller) Summary: memory leaks with new style classes Initial Comment: This code leaks 19 references: class X(object): pass This code leaks 12 references: class X(type): pass ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2001-11-27 11:58 Message: Logged In: YES user_id=11105 Guido is right - it were circular references cleaned up by the garbage collector. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2001-11-24 10:04 Message: Logged In: YES user_id=33168 I just tried both lines with purify and it reported no leaks on Solaris 8. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:13 Message: Logged In: YES user_id=6380 Which version did you try? Which tool did you use? I observed no memory growth for this program (with 2.2b2+): while 1: class C(object): pass class T(type): pass so I conclude there's no leak. There may appear to be a leak, but this is due to circular references and properly taken care of by the garbage collector. So, back to you -- you can either close this, or provide more information and assign it back to me. ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2001-11-23 12:30 Message: Logged In: YES user_id=11105 I forgot: 2.2b2, Windows 2000, debug build. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484977&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:03:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:03:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-485602 ] new module broken with new classes Message-ID: Bugs item #485602, was opened at 2001-11-26 06:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) >Assigned to: Guido van Rossum (gvanrossum) Summary: new module broken with new classes Initial Comment: The new module doesn't support the new-style classes. Example: import new class foo(object): pass a = new.instance(foo,{}) I didn't really expect this to work, but I'm trying to figure out how to emulate the behavior of this function with new-style classes (i.e., how to bring a new-style instance into being without calling __init__). Does a call to PyType_GenericAlloc(PyTypeObject *type, 1) emulate this behavior? Please advise. Cheers, Dave ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:03 Message: Logged In: YES user_id=31435 Assigned to Guido. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 09:37 Message: Logged In: YES user_id=31435 Changed Category & Group. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:04:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:04:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-485152 ] memory leak in test_scope Message-ID: Bugs item #485152, was opened at 2001-11-24 11:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485152&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Barry Warsaw (bwarsaw) Summary: memory leak in test_scope Initial Comment: test_scope leaks memory see attached file for details ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:04 Message: Logged In: YES user_id=31435 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485152&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:04:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:04:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-485150 ] memory leak from marshal.c (test_email) Message-ID: Bugs item #485150, was opened at 2001-11-24 11:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485150&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Barry Warsaw (bwarsaw) Summary: memory leak from marshal.c (test_email) Initial Comment: marshal.c seems to be the culprit in leaking data from running test_email see attached file for details ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:04 Message: Logged In: YES user_id=31435 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485150&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:05:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:05:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-485145 ] memory leak in test_unicode Message-ID: Bugs item #485145, was opened at 2001-11-24 10:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485145&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Barry Warsaw (bwarsaw) Summary: memory leak in test_unicode Initial Comment: test_unicode leaks memory see attached file for details ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:05 Message: Logged In: YES user_id=31435 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485145&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:05:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:05:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-485142 ] memory leak in test_binop Message-ID: Bugs item #485142, was opened at 2001-11-24 10:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485142&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Barry Warsaw (bwarsaw) Summary: memory leak in test_binop Initial Comment: test_binop leaks memory in longobjects see attached file for detailed info ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:05 Message: Logged In: YES user_id=31435 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485142&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:05:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:05:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-485139 ] mem leak in interpreter from syntax err Message-ID: Bugs item #485139, was opened at 2001-11-24 10:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485139&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Barry Warsaw (bwarsaw) Summary: mem leak in interpreter from syntax err Initial Comment: when a syntax error occurs in the interpreter, the interpreter leaks see the attached file for more info ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:05 Message: Logged In: YES user_id=31435 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485139&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:06:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:06:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-485133 ] memory leak in test_parser Message-ID: Bugs item #485133, was opened at 2001-11-24 10:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485133&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) >Assigned to: Barry Warsaw (bwarsaw) Summary: memory leak in test_parser Initial Comment: parsermodule.c leaks in recursive calls to build_node_children(). see attached file for info ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:06 Message: Logged In: YES user_id=31435 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485133&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:07:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:07:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-485602 ] new module broken with new classes Message-ID: Bugs item #485602, was opened at 2001-11-26 06:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) >Assigned to: Nobody/Anonymous (nobody) Summary: new module broken with new classes Initial Comment: The new module doesn't support the new-style classes. Example: import new class foo(object): pass a = new.instance(foo,{}) I didn't really expect this to work, but I'm trying to figure out how to emulate the behavior of this function with new-style classes (i.e., how to bring a new-style instance into being without calling __init__). Does a call to PyType_GenericAlloc(PyTypeObject *type, 1) emulate this behavior? Please advise. Cheers, Dave ---------------------------------------------------------------------- >Comment By: Thomas Heller (theller) Date: 2001-11-27 12:06 Message: Logged In: YES user_id=11105 Doesn't a = foo.__new__(foo) a.__dict__.update(dict) do this? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:03 Message: Logged In: YES user_id=31435 Assigned to Guido. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 09:37 Message: Logged In: YES user_id=31435 Changed Category & Group. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:07:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:07:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) >Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:11:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:11:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-485602 ] new module broken with new classes Message-ID: Bugs item #485602, was opened at 2001-11-26 06:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) >Assigned to: Guido van Rossum (gvanrossum) Summary: new module broken with new classes Initial Comment: The new module doesn't support the new-style classes. Example: import new class foo(object): pass a = new.instance(foo,{}) I didn't really expect this to work, but I'm trying to figure out how to emulate the behavior of this function with new-style classes (i.e., how to bring a new-style instance into being without calling __init__). Does a call to PyType_GenericAlloc(PyTypeObject *type, 1) emulate this behavior? Please advise. Cheers, Dave ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2001-11-27 12:06 Message: Logged In: YES user_id=11105 Doesn't a = foo.__new__(foo) a.__dict__.update(dict) do this? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:03 Message: Logged In: YES user_id=31435 Assigned to Guido. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 09:37 Message: Logged In: YES user_id=31435 Changed Category & Group. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:13:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:13:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-485175 ] buffer overflow in traceback.c Message-ID: Bugs item #485175, was opened at 2001-11-24 12:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485175&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Alex Martelli (aleax) >Assigned to: Tim Peters (tim_one) Summary: buffer overflow in traceback.c Initial Comment: Running this script: name = 'prova'*200 fou = open('fatto.py','w') print>>fou, 'def',name,'():\n return 1/0' fou.close() import fatto print 'prima (%d)'%len(name) funz = getattr(fatto, name) try: funz() except: print 'beccato' raise Python exits with a segfault. Cause: buffer overflow in traceback.c line 157, the only sprintf -- FMT (wrongly, I surmise) does not limit the number of characters it tries to write to linebuf from argument name, an unbounded-length string (the co_name). ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:13 Message: Logged In: YES user_id=31435 Reproduced the problem and assigned to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485175&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:32:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:32:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-485175 ] buffer overflow in traceback.c Message-ID: Bugs item #485175, was opened at 2001-11-24 12:37 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485175&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Alex Martelli (aleax) Assigned to: Tim Peters (tim_one) Summary: buffer overflow in traceback.c Initial Comment: Running this script: name = 'prova'*200 fou = open('fatto.py','w') print>>fou, 'def',name,'():\n return 1/0' fou.close() import fatto print 'prima (%d)'%len(name) funz = getattr(fatto, name) try: funz() except: print 'beccato' raise Python exits with a segfault. Cause: buffer overflow in traceback.c line 157, the only sprintf -- FMT (wrongly, I surmise) does not limit the number of characters it tries to write to linebuf from argument name, an unbounded-length string (the co_name). ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:32 Message: Logged In: YES user_id=31435 Fixed, in Misc/ACKS; new revision: 1.138 Python/traceback.c; new revision: 2.35 The format now limits file and function names to 500 chars each (and the size of the buffer was boosted accordingly). ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:13 Message: Logged In: YES user_id=31435 Reproduced the problem and assigned to me. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485175&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:47:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:47:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-485153 ] Erroneous Fail of PyEval_CallObject Message-ID: Bugs item #485153, was opened at 2001-11-24 11:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 Category: None Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Cl. Schmidt (clemm) >Assigned to: Tim Peters (tim_one) Summary: Erroneous Fail of PyEval_CallObject Initial Comment: If embedding python v2.1.1 in a Windows MFC project, it shows the following behaviour: When a call to e.g. PyArgParseTuple had failed, a subsequent call to PyEval_CallObject fails. The parameters of the both calls are completely independent and have nothing to do with each other. Trying to retrieve the error text of the failed PyEval_CallObject returns the error of the PyArgParseTuple, namely "new style getargs format but argument list is not a tuple". If the first (erroneous) call to PyArgParseTuple is commented, the PyEval_CallObject works without any problem. The attached code snippet documents the error. No attempts have been made to reproduce the behaviour in other environments. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:47 Message: Logged In: YES user_id=31435 I don't understand what "the bug" is here: if you get an error return from a Python C API function, and you intend to ignore the error, you must call PyErr_Clear() before calling another Python C API function. You're not doing that. The attachment isn't executable as-is, so I can't say whether that fixes your particular problem -- but it's never legitimate to ignore an error in C API coding (you must either pass it on to your caller or explicitly clear it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:53:58 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:53:58 -0800 Subject: [Python-bugs-list] [ python-Bugs-484715 ] Upgrade TCL for windows installer Message-ID: Bugs item #484715, was opened at 2001-11-22 17:38 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484715&group_id=5470 Category: Installation >Group: Feature Request Status: Open >Resolution: Postponed Priority: 5 Submitted By: Kirill Simonov (kirill_simonov) Assigned to: Nobody/Anonymous (nobody) Summary: Upgrade TCL for windows installer Initial Comment: Windows installer comes with TCL/TK 8.3.2. The latest version of TCL/TK is 8.3.4. One of the "greatest" features of the latest TCL/TK is the ability to change window icons. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:49 Message: Logged In: YES user_id=31435 This isn't "a bug", but it might be a feature request. Alas, I can't make time to do anything about it, and it's too late for 2.2 regardless. Setting to Postponed until after 2.2. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484715&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:57:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:57:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-484645 ] little bug in pycodegen.py Message-ID: Bugs item #484645, was opened at 2001-11-22 12:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484645&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) >Assigned to: Jeremy Hylton (jhylton) Summary: little bug in pycodegen.py Initial Comment: When the script is called directly (__name__ == "__main__"), compileFile() should be called, not compile() as it is now. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:57 Message: Logged In: YES user_id=31435 Assigned to Jeremy. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484645&group_id=5470 From noreply@sourceforge.net Tue Nov 27 20:55:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 12:55:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-485080 ] lib/test/data not installed Message-ID: Bugs item #485080, was opened at 2001-11-24 02:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485080&group_id=5470 Category: None Group: None Status: Open Resolution: None Priority: 5 Submitted By: Grzegorz Makarewicz (makaron) >Assigned to: Barry Warsaw (bwarsaw) Summary: lib/test/data not installed Initial Comment: test/data should be inserted in LIBSUBDIRS of file Makefile.pre.in since it's not being installed right now. test/test_email fails. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:55 Message: Logged In: YES user_id=31435 Assigned to Barry. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485080&group_id=5470 From noreply@sourceforge.net Tue Nov 27 21:00:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 13:00:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on unbounded recursion in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 4 Submitted By: te (elaias) >Assigned to: Tim Peters (tim_one) Summary: crash on unbounded recursion in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:00 Message: Logged In: YES user_id=31435 Assigned to me (to figure out why the Python-level recursion check isn't triggering, not to tumble into the bottomless "outguess all C runtimes" pit). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:17 Message: Logged In: YES user_id=6380 Let's be clear about rexec. It is currently a pathetic excuse for security. It *could* be made safe, and this is one of the things that would have to be fixed; but there are so many other places that I don't really care any more whether this particular core dump gets fixed (at least not from a security p.o.v.). It's strange though that even though there's a Python function at each level, the Python stack limit check doesn't trigger before C stack overflows. Maybe it would make more sense to spend some time getting the general stack limit check more robust. Unfortunately this has to be done separately for each C compiler and each operating system variation... :-( ---------------------------------------------------------------------- Comment By: Andrew Bennetts (spiv) Date: 2001-11-20 06:53 Message: Logged In: YES user_id=50945 I agree that segfaults like this are a problem; this is a potential security problem because it means that untrusted code can crash the interpreter, even if you try to use something like rexec. ---------------------------------------------------------------------- Comment By: te (elaias) Date: 2001-11-19 09:45 Message: Logged In: YES user_id=109241 Sorry, I thought that this was obvious enough for the person responsible that I didn't have to explain. Thanks for the correction though. And I would say that a segfault in any program is a problem, especially an interpreter. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Tue Nov 27 21:18:40 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 13:18:40 -0800 Subject: [Python-bugs-list] [ python-Bugs-485153 ] Erroneous Fail of PyEval_CallObject Message-ID: Bugs item #485153, was opened at 2001-11-24 11:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 >Category: Documentation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Cl. Schmidt (clemm) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Erroneous Fail of PyEval_CallObject Initial Comment: If embedding python v2.1.1 in a Windows MFC project, it shows the following behaviour: When a call to e.g. PyArgParseTuple had failed, a subsequent call to PyEval_CallObject fails. The parameters of the both calls are completely independent and have nothing to do with each other. Trying to retrieve the error text of the failed PyEval_CallObject returns the error of the PyArgParseTuple, namely "new style getargs format but argument list is not a tuple". If the first (erroneous) call to PyArgParseTuple is commented, the PyEval_CallObject works without any problem. The attached code snippet documents the error. No attempts have been made to reproduce the behaviour in other environments. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:14 Message: Logged In: YES user_id=31435 Changed to Documentation and reassigned to Fred: Fred, I don't think we ever spell out that C API errors must be passed on or explicitly cleared (before calling another C API function). The Exceptions section of the C API manual does not spell this out. It's possible that the "Intermezzo: Errors and Exceptions" section of the Extending and Embedding manual is clear enough -- your call. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:47 Message: Logged In: YES user_id=31435 I don't understand what "the bug" is here: if you get an error return from a Python C API function, and you intend to ignore the error, you must call PyErr_Clear() before calling another Python C API function. You're not doing that. The attachment isn't executable as-is, so I can't say whether that fixes your particular problem -- but it's never legitimate to ignore an error in C API coding (you must either pass it on to your caller or explicitly clear it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 From noreply@sourceforge.net Tue Nov 27 21:33:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 13:33:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-475877 ] Mutable subtype instances are hashable Message-ID: Bugs item #475877, was opened at 2001-10-28 19:24 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Tim Peters (tim_one) >Assigned to: Guido van Rossum (gvanrossum) Summary: Mutable subtype instances are hashable Initial Comment: >>> class D(dictionary): pass ... >>> d = D() >>> hash(d) 7920544 >>> id(d) 7920544 >>> Ditto for instances of list subclasses: >>> class L(list): pass ... >>> x = L(range(100)) >>> hash(x) 7928992 >>> id(x) 7928992 >>> Among other nasties, this lets them get used as mutable dict keys. Reported by Mark J on c.l.py: """ From: Mark J Sent: Sunday, October 28, 2001 10:03 PM To: python-list@python.org Subject: Python 2.2b1 hashable dictionary bug? Since I haven't had a good hit rate at detecting bugs vs. features, I thought I'd post here before filing a bug report at SourceForge. Python 2.2b1 (#1, Oct 19 2001, 23:11:09) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class D(dictionary): pass ... >>> d = {} >>> d2 = D() >>> d[d2] = "dictionary used as key" >>> d {{}: 'dictionary used as key'} >>> d[D()]="now have two keys that look the same" >>> d {{}: 'now have two keys that look the same', {}: 'dictionary used as key'} >>> d2["key"] = "mutable key in d" >>> d {{}: 'now have two keys that look the same', {'key': 'mutable key in d'}: 'dictionary used as key'} >>> d[d] = "plain dictionary not allowed as key" Traceback (most recent call last): File "", line 1, in ? TypeError: unhashable type >>> d2[d2] = "subclassed dictionary allowed though" >>> d2 {{...}: 'subclassed dictionary allowed though', 'key': 'mutable keys'} >>> #whoa: what just happened: {...} ?? Interesting.... So is this a feature or a bug? """ ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:33 Message: Logged In: YES user_id=31435 Assigned to Guido, since he admitted to thinking about it. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-29 11:39 Message: Logged In: YES user_id=6380 Interestingly, in Python list.__hash__ is the same as object.__hash__, but in C, list.tp_hash is NULL while object.tp_hash is not. I think that the best solution is to somehow program an exception into add_operators that adds a dummy __hash__ wrapper (which always raises an exception) when the tp_hash field is found to be NULL. (Note that inherit_slots already contains special-casing for tp_hash.) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-28 23:36 Message: Logged In: YES user_id=6380 There's an admin setting that auto-assigns certain categories. Documentation gets auto-assugned to Fred; type/class to me; Regular Expressions to Effbot I believe. The hash(sys.stdin) result is expected; files are compared by address and so their hash() is derived from their address too. I'll think about the real issue; this has to do with the way the hash stub gets set. I'm not sure yet whether this is best fixed by adding more code to slot_tp_hash or to the code that sticks slot_to_hash in the tp_hash slot. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:33 Message: Logged In: YES user_id=31435 I swear I didn't assign this to Guido -- I intended to leave this unassigned for now. That's the second time in two weeks I believe SF made up an assignment on one of my reports. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-10-28 19:28 Message: Logged In: YES user_id=31435 Jeez, hash() has gone off the deep end: >>> import sys >>> hash(sys.stdin) 7690032 >>> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=475877&group_id=5470 From noreply@sourceforge.net Tue Nov 27 21:36:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 13:36:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-478768 ] type attribute hides member of same name Message-ID: Bugs item #478768, was opened at 2001-11-06 08:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Michael McLay (mclay) >Assigned to: Guido van Rossum (gvanrossum) Summary: type attribute hides member of same name Initial Comment: Allowing type attributes to replace member descriptors of the same name removes access to all instance data of the same name throughout the application. The error generated is not obvious and may not occur near the source of the error. A patch to prevent this is attached. The use of __slots__ to define attributes allowed in an instance of a type eliminates the __dict__ in instance. Instead the names of the instance attributes are declared in the __slots__ attribute of the class definition. The names defined in __slots__ become member descriptor in the type dict. This change has useful advantages, but there is one side effect that makes the use of __slots__ different from the semantics of the old style dynamic classes. It is not possible to have a type attribute and an instance attribute with the same name. This eliminates the idiom of using a class attribute as a default and then overriding the default value by creating an instance attribute of the same name. This is no longer possible because the declaration of a type attribute name will replace the definition of the member name in the type dict. Once this occurs it impossible to access any instances of the attribute of the same name as a type attribute because the set and get methods for accessing the data are held in the member descriptor. The AttributeError raised in the following example illustrates the problem with using the names declared in __slots__ when naming a type attribute. >>> class B(object): ... a = 3 ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> The member descriptor 'a' defined in __slots__ is inaccessible because the type dict has overwritten the member descriptor with the type attribute of the same name. A more descriptive error message can be generated by checking that no __slots__ names are in the type dict when the __slots__ are being created by type_new(). An instance member defined by __slots__ can also be hidden by a type attribute following the completion of the definition of the class by making an assignment to the type with the same name as the instance member. In the following example the "B.a = "hiding b.a" replaces the reference to the member descriptor for instance member 'a' in the type dict. This eliminates access to all instance members of that name throughout the application. >>> class B(object): ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 >>> B.a = "hiding b.a" >>> b.a 'hiding b.a' >>> b.a = 5 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:35 Message: Logged In: YES user_id=31435 Reassigned to Guido. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-06 09:43 Message: Logged In: YES user_id=31435 Changed category, and assigned to me in Guido's absence. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 From noreply@sourceforge.net Tue Nov 27 22:37:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 14:37:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on unbounded recursion in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 4 Submitted By: te (elaias) >Assigned to: Nobody/Anonymous (nobody) Summary: crash on unbounded recursion in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 14:37 Message: Logged In: YES user_id=31435 OK, Python's recursion check does trigger if "del c" is included at the end of the __del__ method. However, as-is, the destruction of c is a side effect of __del__'s frame getting decref'ed, and the latter happens at the end of PyEval_EvalCodeEx(). eval_frame() is long gone by then, and tstate->recursion_depth is checked in the latter. To be clear, there is a deep chain of Python frames, but they're all in the process of being destructed, and tstate- >recursion_depth has already been decremented (it's just 3 at the time this blew up on my Windows box): eval_frame() is not on the C stack (except for a few times at the base of the C stack). So, in this sense, it's a failure of the recursion-depth hacks to model the true depth of the C stack. Note there are actually 10 levels of C stack for each Python level here: PyEval_EvalCodeEx decrefs the frame, which invokes _Py_Dealloc, then to frame_dealloc, which decrefs c and triggers _Py_Dealloc -> instance_dealloc, which finds a __del__ method and so does PyEval_CallObjectWithKeywords -> PyObject_Call -> instancemethod_call -> PyObject_Call -> function_call -> PyEval_EvalCodeEx, and we start all over again. No bright ideas here, so unassigned again. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:00 Message: Logged In: YES user_id=31435 Assigned to me (to figure out why the Python-level recursion check isn't triggering, not to tumble into the bottomless "outguess all C runtimes" pit). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:17 Message: Logged In: YES user_id=6380 Let's be clear about rexec. It is currently a pathetic excuse for security. It *could* be made safe, and this is one of the things that would have to be fixed; but there are so many other places that I don't really care any more whether this particular core dump gets fixed (at least not from a security p.o.v.). It's strange though that even though there's a Python function at each level, the Python stack limit check doesn't trigger before C stack overflows. Maybe it would make more sense to spend some time getting the general stack limit check more robust. Unfortunately this has to be done separately for each C compiler and each operating system variation... :-( ---------------------------------------------------------------------- Comment By: Andrew Bennetts (spiv) Date: 2001-11-20 06:53 Message: Logged In: YES user_id=50945 I agree that segfaults like this are a problem; this is a potential security problem because it means that untrusted code can crash the interpreter, even if you try to use something like rexec. ---------------------------------------------------------------------- Comment By: te (elaias) Date: 2001-11-19 09:45 Message: Logged In: YES user_id=109241 Sorry, I thought that this was obvious enough for the person responsible that I didn't have to explain. Thanks for the correction though. And I would say that a segfault in any program is a problem, especially an interpreter. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Tue Nov 27 23:03:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 15:03:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on unbounded recursion in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 Status: Open Resolution: None Priority: 4 Submitted By: te (elaias) >Assigned to: Tim Peters (tim_one) Summary: crash on unbounded recursion in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 15:03 Message: Logged In: YES user_id=31435 Back to me. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 14:37 Message: Logged In: YES user_id=31435 OK, Python's recursion check does trigger if "del c" is included at the end of the __del__ method. However, as-is, the destruction of c is a side effect of __del__'s frame getting decref'ed, and the latter happens at the end of PyEval_EvalCodeEx(). eval_frame() is long gone by then, and tstate->recursion_depth is checked in the latter. To be clear, there is a deep chain of Python frames, but they're all in the process of being destructed, and tstate- >recursion_depth has already been decremented (it's just 3 at the time this blew up on my Windows box): eval_frame() is not on the C stack (except for a few times at the base of the C stack). So, in this sense, it's a failure of the recursion-depth hacks to model the true depth of the C stack. Note there are actually 10 levels of C stack for each Python level here: PyEval_EvalCodeEx decrefs the frame, which invokes _Py_Dealloc, then to frame_dealloc, which decrefs c and triggers _Py_Dealloc -> instance_dealloc, which finds a __del__ method and so does PyEval_CallObjectWithKeywords -> PyObject_Call -> instancemethod_call -> PyObject_Call -> function_call -> PyEval_EvalCodeEx, and we start all over again. No bright ideas here, so unassigned again. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:00 Message: Logged In: YES user_id=31435 Assigned to me (to figure out why the Python-level recursion check isn't triggering, not to tumble into the bottomless "outguess all C runtimes" pit). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:17 Message: Logged In: YES user_id=6380 Let's be clear about rexec. It is currently a pathetic excuse for security. It *could* be made safe, and this is one of the things that would have to be fixed; but there are so many other places that I don't really care any more whether this particular core dump gets fixed (at least not from a security p.o.v.). It's strange though that even though there's a Python function at each level, the Python stack limit check doesn't trigger before C stack overflows. Maybe it would make more sense to spend some time getting the general stack limit check more robust. Unfortunately this has to be done separately for each C compiler and each operating system variation... :-( ---------------------------------------------------------------------- Comment By: Andrew Bennetts (spiv) Date: 2001-11-20 06:53 Message: Logged In: YES user_id=50945 I agree that segfaults like this are a problem; this is a potential security problem because it means that untrusted code can crash the interpreter, even if you try to use something like rexec. ---------------------------------------------------------------------- Comment By: te (elaias) Date: 2001-11-19 09:45 Message: Logged In: YES user_id=109241 Sorry, I thought that this was obvious enough for the person responsible that I didn't have to explain. Thanks for the correction though. And I would say that a segfault in any program is a problem, especially an interpreter. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Tue Nov 27 23:08:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 15:08:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Tue Nov 27 23:21:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 15:21:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-27 15:21 Message: Logged In: YES user_id=85984 I probably should have included the output from gcc 2.7.2.1 instead which I know is properly installed. The results are similar, although there is no mention of `wint_t': % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from Include/pyport.h:93, from Include/Python.h:60, from ./Modules/socketmodule.c:78: /usr/include/math.h:177: warning: function declaration isn't a prototype /usr/include/math.h:257: warning: function declaration isn't a prototype In file included from Include/unicodeobject.h:118, from Include/Python.h:69, from ./Modules/socketmodule.c:78: /usr/include/wchar.h:15: warning: function declaration isn't a prototype In file included from /usr/include/signal.h:46, from ./Modules/socketmodule.c:140: /usr/include/sys/signal.h:121: warning: function declaration isn't a prototype /usr/include/sys/signal.h:164: warning: function declaration isn't a prototype Modules/getaddrinfo.c: In function `gai_strerror': In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Tue Nov 27 23:30:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 15:30:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-486247 ] ftplib/URLLib time outs on transfers Message-ID: Bugs item #486247, was opened at 2001-11-27 15:30 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486247&group_id=5470 Category: Extension Modules Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Tom Fillmore (tfillmor) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib/URLLib time outs on transfers Initial Comment: Hi - On upgrading to v2.1.1 my ftp apps stopped working, all with time out errors when transferring files of any length. I did set the set_pasv method to false, to emulate the v2.0 behavior, which worked OK, but still got the same time out errors. I also tested them with v2.2b but got the same errors. Bottom line - with versions above v2.0 the urllib and ftp;ib extensions time out on transfers. Simply re-installing v2.0 fixed the problem. It is always possible that I was not setting the method correctly, so here's what I did to the set_pasv method: sessionname.set_pasv("false") My specs are: Win2000 server python v2.01 Nothing. There is no third thing... 8-) Thanks in advance! Tom Fillmore tfillmor@oc-bahai.org ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486247&group_id=5470 From noreply@sourceforge.net Tue Nov 27 23:35:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 15:35:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-484645 ] little bug in pycodegen.py Message-ID: Bugs item #484645, was opened at 2001-11-22 12:07 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484645&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Jeremy Hylton (jhylton) Summary: little bug in pycodegen.py Initial Comment: When the script is called directly (__name__ == "__main__"), compileFile() should be called, not compile() as it is now. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:57 Message: Logged In: YES user_id=31435 Assigned to Jeremy. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484645&group_id=5470 From noreply@sourceforge.net Tue Nov 27 23:36:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 15:36:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-483469 ] crash on unbounded recursion in __del__ . Message-ID: Bugs item #483469, was opened at 2001-11-19 07:49 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 Category: Python Interpreter Core Group: Python 2.1.1 >Status: Closed >Resolution: Fixed Priority: 4 Submitted By: te (elaias) Assigned to: Tim Peters (tim_one) Summary: crash on unbounded recursion in __del__ . Initial Comment: do the following. >>> class C: ... def __del__(self): ... c = C() >>> c = C() >>> d = range(100) #anything really >>> c = d >>> c Segmentation fault (core dumped) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 15:36 Message: Logged In: YES user_id=31435 I repaired the tstate->recursion_depth counter for this case, in Python/ceval.c; new revision: 2.291 It doesn't help much, though -- there are so many C stack frames per Python level here that sys.setrecursionlimit() has to be cranked way down from the default to do any good. Even if you do that, the resulting "maximum recursion depth exceeded" exception occurs in a destructor so is ignored (you get a msg on a stderr, but the exception doesn't propagate outside the __del__ method -- note that this is true of *any* exception raised when in a __del__). Marking this Fixed since it's the best we can do with what we've got, and concerned users can force sys.setrecursionlimit() to a low value. I'm not opposed to schemes to guess stack limits in some other way, but discussion of that doesn't belong in this bug report, and we've already got the platform-dependent PyOS_CheckStack() gimmick that doesn't appear to be worth having anyway. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 15:03 Message: Logged In: YES user_id=31435 Back to me. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 14:37 Message: Logged In: YES user_id=31435 OK, Python's recursion check does trigger if "del c" is included at the end of the __del__ method. However, as-is, the destruction of c is a side effect of __del__'s frame getting decref'ed, and the latter happens at the end of PyEval_EvalCodeEx(). eval_frame() is long gone by then, and tstate->recursion_depth is checked in the latter. To be clear, there is a deep chain of Python frames, but they're all in the process of being destructed, and tstate- >recursion_depth has already been decremented (it's just 3 at the time this blew up on my Windows box): eval_frame() is not on the C stack (except for a few times at the base of the C stack). So, in this sense, it's a failure of the recursion-depth hacks to model the true depth of the C stack. Note there are actually 10 levels of C stack for each Python level here: PyEval_EvalCodeEx decrefs the frame, which invokes _Py_Dealloc, then to frame_dealloc, which decrefs c and triggers _Py_Dealloc -> instance_dealloc, which finds a __del__ method and so does PyEval_CallObjectWithKeywords -> PyObject_Call -> instancemethod_call -> PyObject_Call -> function_call -> PyEval_EvalCodeEx, and we start all over again. No bright ideas here, so unassigned again. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:00 Message: Logged In: YES user_id=31435 Assigned to me (to figure out why the Python-level recursion check isn't triggering, not to tumble into the bottomless "outguess all C runtimes" pit). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-23 18:17 Message: Logged In: YES user_id=6380 Let's be clear about rexec. It is currently a pathetic excuse for security. It *could* be made safe, and this is one of the things that would have to be fixed; but there are so many other places that I don't really care any more whether this particular core dump gets fixed (at least not from a security p.o.v.). It's strange though that even though there's a Python function at each level, the Python stack limit check doesn't trigger before C stack overflows. Maybe it would make more sense to spend some time getting the general stack limit check more robust. Unfortunately this has to be done separately for each C compiler and each operating system variation... :-( ---------------------------------------------------------------------- Comment By: Andrew Bennetts (spiv) Date: 2001-11-20 06:53 Message: Logged In: YES user_id=50945 I agree that segfaults like this are a problem; this is a potential security problem because it means that untrusted code can crash the interpreter, even if you try to use something like rexec. ---------------------------------------------------------------------- Comment By: te (elaias) Date: 2001-11-19 09:45 Message: Logged In: YES user_id=109241 Sorry, I thought that this was obvious enough for the person responsible that I didn't have to explain. Thanks for the correction though. And I would say that a segfault in any program is a problem, especially an interpreter. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-19 09:18 Message: Logged In: YES user_id=31435 Well, it's not just "an assignment": the __del__ creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, which creates a local instance of C, which is destroyed while __del__ is trying to return, which calls __del__ again, etc etc etc. So it's roughly the same as class C: def __str__(self): return str(self) print C() You get unbounded recursion and eventually the stack blows up. "Don't do that" is the best advice . Reduced priority accordingly; *maybe* Python can be changed to detect the stack overflow and give a "maximum recursion depth" exception instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483469&group_id=5470 From noreply@sourceforge.net Tue Nov 27 23:41:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 15:41:38 -0800 Subject: [Python-bugs-list] [ python-Bugs-476852 ] Some bad macros in abstract.h Message-ID: Bugs item #476852, was opened at 2001-10-31 10:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476852&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Guido van Rossum (gvanrossum) >Assigned to: Jeremy Hylton (jhylton) Summary: Some bad macros in abstract.h Initial Comment: Greg Chapman reported in c.l.py: abstract.h defines the following: #define PyMapping_DelItemString(O,K) PyDict_DelItemString((O),(K)) #define PyMapping_DelItem(O,K) PyDict_DelItem((O),(K)) It seems to me they should delegate to PyObject_DelItem (and PyObject_DelItemString, which will have to be added). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476852&group_id=5470 From noreply@sourceforge.net Wed Nov 28 01:17:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 17:17:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-486278 ] SystemError: Python/getargs.c:1086: bad Message-ID: Bugs item #486278, was opened at 2001-11-27 17:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: SystemError: Python/getargs.c:1086: bad Initial Comment: This looks like a new problem introduced in 2.2b2 (or something has changed in the way we are to write extention modules in C)...I'm using Solaris 2.6 with no configure options other then --prefix=... I started with two fresh builds and copied the same files their proper places. One was Python-2.2b1 and the other was Python-2.2b2. My extention module is fairly a fairly simple conversion utility. Both builds complete cleanly. Here is the output from 2.2b1: ----------------------- Python 2.2b1 (#1, Nov 27 2001, 19:28:20) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import Cstimbase >>> Cstimbase.hex2wire('aaff11') '\xaa\xff\x11' >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) 'aaff11' >>> --------------------------- Here is the exact same module running in 2.2b2: ------------------------------- Python 2.2b2 (#1, Nov 27 2001, 16:42:01) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import Cstimbase >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) Traceback (most recent call last): File "", line 1, in ? SystemError: Python/getargs.c:1086: bad argument to internal function >>> ----------------------------- So what changed in the way you write C extention modules? Thanks, Sam (stannous@employees.org) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 From noreply@sourceforge.net Wed Nov 28 02:38:31 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 18:38:31 -0800 Subject: [Python-bugs-list] [ python-Bugs-485602 ] new module broken with new classes Message-ID: Bugs item #485602, was opened at 2001-11-26 06:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: David M. Beazley (beazley) Assigned to: Guido van Rossum (gvanrossum) Summary: new module broken with new classes Initial Comment: The new module doesn't support the new-style classes. Example: import new class foo(object): pass a = new.instance(foo,{}) I didn't really expect this to work, but I'm trying to figure out how to emulate the behavior of this function with new-style classes (i.e., how to bring a new-style instance into being without calling __init__). Does a call to PyType_GenericAlloc(PyTypeObject *type, 1) emulate this behavior? Please advise. Cheers, Dave ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-27 18:38 Message: Logged In: YES user_id=6380 Thomas is right on the money: someclass.__new__() should do it. PyType_GenericAlloc() may create an instance that isn't sufficiently initialized to be used safely (it initializes everything to zeros, but that's not enough e.g. for dict instances). ---------------------------------------------------------------------- Comment By: Thomas Heller (theller) Date: 2001-11-27 12:06 Message: Logged In: YES user_id=11105 Doesn't a = foo.__new__(foo) a.__dict__.update(dict) do this? ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:03 Message: Logged In: YES user_id=31435 Assigned to Guido. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 09:37 Message: Logged In: YES user_id=31435 Changed Category & Group. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485602&group_id=5470 From noreply@sourceforge.net Wed Nov 28 02:46:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 18:46:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-486144 ] Uninitialized __slot__ vrbl is None Message-ID: Bugs item #486144, was opened at 2001-11-27 11:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486144&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Guido van Rossum (gvanrossum) Summary: Uninitialized __slot__ vrbl is None Initial Comment: >>> class C(object): ... __slots__ = ['a'] ... >>> c = C() >>> print c.a None >>> Nothing else in Python works like this -- a (possibly subclass of) NameError exception would be better. ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-27 18:45 Message: Logged In: YES user_id=6380 While this is not normal for class instances, built-in types have long had this behavior -- if a built-in type has an object pointer attribute described by a structmember with code T_OBJECT, it has this behavior (NULL is treated as None rather than absent attribute). I propose to add a T_OBJECT_EX structmember code, which raises an exception for NULL instead of returning None. This can then be used by the __slots__ code in typeobject.c. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486144&group_id=5470 From noreply@sourceforge.net Wed Nov 28 03:02:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 19:02:20 -0800 Subject: [Python-bugs-list] [ python-Bugs-478768 ] type attribute hides member of same name Message-ID: Bugs item #478768, was opened at 2001-11-06 08:53 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 Category: Type/class unification Group: Python 2.2 >Status: Closed >Resolution: Rejected Priority: 5 Submitted By: Michael McLay (mclay) Assigned to: Guido van Rossum (gvanrossum) Summary: type attribute hides member of same name Initial Comment: Allowing type attributes to replace member descriptors of the same name removes access to all instance data of the same name throughout the application. The error generated is not obvious and may not occur near the source of the error. A patch to prevent this is attached. The use of __slots__ to define attributes allowed in an instance of a type eliminates the __dict__ in instance. Instead the names of the instance attributes are declared in the __slots__ attribute of the class definition. The names defined in __slots__ become member descriptor in the type dict. This change has useful advantages, but there is one side effect that makes the use of __slots__ different from the semantics of the old style dynamic classes. It is not possible to have a type attribute and an instance attribute with the same name. This eliminates the idiom of using a class attribute as a default and then overriding the default value by creating an instance attribute of the same name. This is no longer possible because the declaration of a type attribute name will replace the definition of the member name in the type dict. Once this occurs it impossible to access any instances of the attribute of the same name as a type attribute because the set and get methods for accessing the data are held in the member descriptor. The AttributeError raised in the following example illustrates the problem with using the names declared in __slots__ when naming a type attribute. >>> class B(object): ... a = 3 ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> The member descriptor 'a' defined in __slots__ is inaccessible because the type dict has overwritten the member descriptor with the type attribute of the same name. A more descriptive error message can be generated by checking that no __slots__ names are in the type dict when the __slots__ are being created by type_new(). An instance member defined by __slots__ can also be hidden by a type attribute following the completion of the definition of the class by making an assignment to the type with the same name as the instance member. In the following example the "B.a = "hiding b.a" replaces the reference to the member descriptor for instance member 'a' in the type dict. This eliminates access to all instance members of that name throughout the application. >>> class B(object): ... __slots__ = ['a'] ... >>> b = B() >>> b.a = 4 >>> B.a = "hiding b.a" >>> b.a 'hiding b.a' >>> b.a = 5 Traceback (most recent call last): File "", line 1, in ? AttributeError: 'B' object attribute 'a' is read-only >>> ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-27 19:02 Message: Logged In: YES user_id=6380 Rejected. The patch would make it impossible to replace a descriptor. __slots__ needs documentation. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:35 Message: Logged In: YES user_id=31435 Reassigned to Guido. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-06 09:43 Message: Logged In: YES user_id=31435 Changed category, and assigned to me in Guido's absence. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=478768&group_id=5470 From noreply@sourceforge.net Wed Nov 28 03:49:22 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 19:49:22 -0800 Subject: [Python-bugs-list] [ python-Bugs-486278 ] SystemError: Python/getargs.c:1086: bad Message-ID: Bugs item #486278, was opened at 2001-11-27 17:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: SystemError: Python/getargs.c:1086: bad Initial Comment: This looks like a new problem introduced in 2.2b2 (or something has changed in the way we are to write extention modules in C)...I'm using Solaris 2.6 with no configure options other then --prefix=... I started with two fresh builds and copied the same files their proper places. One was Python-2.2b1 and the other was Python-2.2b2. My extention module is fairly a fairly simple conversion utility. Both builds complete cleanly. Here is the output from 2.2b1: ----------------------- Python 2.2b1 (#1, Nov 27 2001, 19:28:20) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import Cstimbase >>> Cstimbase.hex2wire('aaff11') '\xaa\xff\x11' >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) 'aaff11' >>> --------------------------- Here is the exact same module running in 2.2b2: ------------------------------- Python 2.2b2 (#1, Nov 27 2001, 16:42:01) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import Cstimbase >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) Traceback (most recent call last): File "", line 1, in ? SystemError: Python/getargs.c:1086: bad argument to internal function >>> ----------------------------- So what changed in the way you write C extention modules? Thanks, Sam (stannous@employees.org) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-27 19:49 Message: Logged In: YES user_id=31435 Do you call PyArg_ParseTupleAndKeywords() in your extension? The error you're seeing is a new sanity check, verifying that the number of keyword args passed in kwlist is no larger than the number of argument codes passed in the format string. It didn't used to check this, and it was possible for Python to dump core as a result. Please check all your uses of ParseTupleAndKeywords() -- I can't do it for you without seeing your code. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 From noreply@sourceforge.net Wed Nov 28 05:35:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 21:35:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-486360 ] Support for page & shortcut icons Message-ID: Bugs item #486360, was opened at 2001-11-27 21:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486360&group_id=5470 Category: Documentation Group: Feature Request Status: Open Resolution: None Priority: 3 Submitted By: Fred L. Drake, Jr. (fdrake) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Support for page & shortcut icons Initial Comment: The generated Python documentation should include support for "page" and "shortcut" icons. These are supported (at least) by MSIE and Mozilla 0.9.6 ("page" only for now, but "shortcut" expected for 0.9.7). Netscape 6.x will acquire support from the Mozilla project. More information: http://www.mozilla.org/releases/mozilla0.9.6/ http://msdn.microsoft.com/workshop/Author/dhtml/howto/ShortcutIcon.asp ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486360&group_id=5470 From noreply@sourceforge.net Wed Nov 28 05:49:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 21:49:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-482003 ] dict display broken in pprint Message-ID: Bugs item #482003, was opened at 2001-11-15 00:26 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482003&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Barry Warsaw (bwarsaw) Summary: dict display broken in pprint Initial Comment: pprint.PrettyPrinter.__format hoses display of multi- line dicts. Instead of placing the ":" after the key, it is placed before the key on continuation lines. The attached patch fixes this bug and adds a test case. Barry, once you're done with this you might want to reassign it to whoever is doing 2.1.2 in case this bug is present there as well. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-27 21:49 Message: Logged In: YES user_id=12800 Applied to pprint.py 1.19 and test_pprint.py 1.7 ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=482003&group_id=5470 From noreply@sourceforge.net Wed Nov 28 06:19:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 22:19:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-484950 ] docs suggest no cyclic garbage collectio Message-ID: Bugs item #484950, was opened at 2001-11-23 10:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 Category: Documentation Group: None >Status: Closed Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: docs suggest no cyclic garbage collectio Initial Comment: http://python.sourceforge.net/devel-docs/ext/refcounts.html Section 1.10 is a bit out of date -- it would lead some readers to believe that Python doesn't collect cyclical garbage. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-27 22:19 Message: Logged In: YES user_id=3066 I must be missing something; what makes you think this implies cyclical garbage isn't collected? Marked pending awaiting further information. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 From noreply@sourceforge.net Wed Nov 28 06:28:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 22:28:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-484950 ] docs suggest no cyclic garbage collectio Message-ID: Bugs item #484950, was opened at 2001-11-23 10:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 Category: Documentation Group: None >Status: Pending Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: docs suggest no cyclic garbage collectio Initial Comment: http://python.sourceforge.net/devel-docs/ext/refcounts.html Section 1.10 is a bit out of date -- it would lead some readers to believe that Python doesn't collect cyclical garbage. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-27 22:28 Message: Logged In: YES user_id=3066 Oops, that was *pending*, not closed! ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-27 22:19 Message: Logged In: YES user_id=3066 I must be missing something; what makes you think this implies cyclical garbage isn't collected? Marked pending awaiting further information. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 From noreply@sourceforge.net Wed Nov 28 07:52:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 23:52:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-485342 ] fcntl.lockf clarification Message-ID: Bugs item #485342, was opened at 2001-11-25 09:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485342&group_id=5470 >Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Skip Montanaro (montanaro) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: fcntl.lockf clarification Initial Comment: fcntl.lockf appears broken to me. It always gives me "bad file descriptor", even though I can use the file object from which it is derived: >>> import fcntl >>> f = open("names.db", "r") >>> fcntl.lockf(f.fileno(), fcntl.LOCK_EX) Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 9] Bad file descriptor >>> data = f.read() >>> len(data) 45056 Since posixfile is officially deprecated, this would be nice to have working... Environment: Mandrake 8.1, gcc 2.96, glibc 2.2.4 ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-27 23:52 Message: Logged In: YES user_id=3066 Fixed by adding the suggested text to Doc/lib/libfcntl.tex revision 1.28. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-11-25 09:13 Message: Logged In: YES user_id=44345 I retract my contention that there is a bug in lockf. It appears that you can't ask for an exclusive lock on a file which is open for read access. As it's not clear from either the fcntl module documentation or the fcntl, lockf or flock man pages (just what does fcntl.lockf emulate by the way?), perhaps something simple should be added to the fcntl.lockf description. Suggestion: On at least some systems, LOCK_EX can only be used if the file descriptor refers to a file opened for writing. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485342&group_id=5470 From noreply@sourceforge.net Wed Nov 28 07:53:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Tue, 27 Nov 2001 23:53:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-483805 ] mmap docs: Minor description problem Message-ID: Bugs item #483805, was opened at 2001-11-20 06:55 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483805&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) >Summary: mmap docs: Minor description problem Initial Comment: Subject: Re: Mutable strings Date: Sun, 18 Nov 2001 22:45:30 -0500 From: "Fred L. Drake, Jr." To: "Colin J. Williams" CC: Python-docs@python.org References: 1 Colin J. Williams writes: > Section 7.7 of the library (mmap) refers to mutable strings. > > At first glance, this is jarring. We learn early on that strings are > immutable. Please file a bug report at SourceForge for this so I don't lose track of it; I'm sure it won't be a difficult fix, but I can't work on it right now. http://sourceforge.net/projects/python/ > Finally, a little grumble about Active State's conversion of the > docs from HTML to the Microsoft world. Among other things, the > links on the About page force one to use the MS Mailer. My default > browser/email facility is Netscape. We don't have any control over ActiveState's packaging; you'll need to refer to their Web site to find who should hear about this problem. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=483805&group_id=5470 From noreply@sourceforge.net Wed Nov 28 09:30:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 01:30:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 01:30 Message: Logged In: YES user_id=21627 Ok, then we need to investigate the configure results. Can you please attach pyconfig.h and config.log to this report? Also, can you please confirm that you build Python from scratch with 2.7.2.1 (i.e untarring a fresh source tree, to avoid using a config.cache left over from gcc 2.95.2)? Furthermore, can you please report details of the getaddrinfo implementation on your system? Does it provide one? In the C library? If not, can you confirm that EAI_ADDRFAMILY is defined in your system headers, but EAI_MAX is not? If yes, can you speculate as to why configure thinks your system does not have getaddrinfo? ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-27 15:21 Message: Logged In: YES user_id=85984 I probably should have included the output from gcc 2.7.2.1 instead which I know is properly installed. The results are similar, although there is no mention of `wint_t': % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from Include/pyport.h:93, from Include/Python.h:60, from ./Modules/socketmodule.c:78: /usr/include/math.h:177: warning: function declaration isn't a prototype /usr/include/math.h:257: warning: function declaration isn't a prototype In file included from Include/unicodeobject.h:118, from Include/Python.h:69, from ./Modules/socketmodule.c:78: /usr/include/wchar.h:15: warning: function declaration isn't a prototype In file included from /usr/include/signal.h:46, from ./Modules/socketmodule.c:140: /usr/include/sys/signal.h:121: warning: function declaration isn't a prototype /usr/include/sys/signal.h:164: warning: function declaration isn't a prototype Modules/getaddrinfo.c: In function `gai_strerror': In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Wed Nov 28 11:25:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 03:25:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-485153 ] Erroneous Fail of PyEval_CallObject Message-ID: Bugs item #485153, was opened at 2001-11-24 11:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 Category: Documentation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Cl. Schmidt (clemm) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Erroneous Fail of PyEval_CallObject Initial Comment: If embedding python v2.1.1 in a Windows MFC project, it shows the following behaviour: When a call to e.g. PyArgParseTuple had failed, a subsequent call to PyEval_CallObject fails. The parameters of the both calls are completely independent and have nothing to do with each other. Trying to retrieve the error text of the failed PyEval_CallObject returns the error of the PyArgParseTuple, namely "new style getargs format but argument list is not a tuple". If the first (erroneous) call to PyArgParseTuple is commented, the PyEval_CallObject works without any problem. The attached code snippet documents the error. No attempts have been made to reproduce the behaviour in other environments. ---------------------------------------------------------------------- >Comment By: Cl. Schmidt (clemm) Date: 2001-11-28 03:25 Message: Logged In: YES user_id=382038 Maybe I violated a python coding rule. In my case, I am embedding python in a windows C++ program. I am having neither a console window nor a caller to the method where the error occurred. Thus, raising an exception would not have made any sense. It is not correct to state that I ignore the error, because there is a return value from the function which is of course retrieved. It is at least disturbing when after having e.g. forgotten to clear an error, three subsequent calls to the C API succeed (as in my case), and suddenly one fails, reporting something completely irrelevant to that error. This behaviour is different in other cases. In another, very similar scenario, the call to PyEval_CallObject succeeded. I agree that it is a documentation issue. The doc should state something like this: "If you do not raise an exception to python and do not call PyErr_Clear, the results are unpredictable." ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 13:14 Message: Logged In: YES user_id=31435 Changed to Documentation and reassigned to Fred: Fred, I don't think we ever spell out that C API errors must be passed on or explicitly cleared (before calling another C API function). The Exceptions section of the C API manual does not spell this out. It's possible that the "Intermezzo: Errors and Exceptions" section of the Extending and Embedding manual is clear enough -- your call. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:47 Message: Logged In: YES user_id=31435 I don't understand what "the bug" is here: if you get an error return from a Python C API function, and you intend to ignore the error, you must call PyErr_Clear() before calling another Python C API function. You're not doing that. The attachment isn't executable as-is, so I can't say whether that fixes your particular problem -- but it's never legitimate to ignore an error in C API coding (you must either pass it on to your caller or explicitly clear it). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485153&group_id=5470 From noreply@sourceforge.net Wed Nov 28 11:51:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 03:51:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c Message-ID: Bugs item #486434, was opened at 2001-11-28 03:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: M.-A. Lemburg (lemburg) Assigned to: Martin v. Löwis (loewis) Summary: Compiler complaints in posixmodule.c Initial Comment: The linker on Linux reports some warnings for posixmodule.c: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436: the use of `tempnam' is dangerous, better use `mkstemp' Perhaps we ought to follow the advice ?! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 From noreply@sourceforge.net Wed Nov 28 11:52:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 03:52:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-485951 ] repr diff between string and unicode. Message-ID: Bugs item #485951, was opened at 2001-11-27 02:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 Category: Unicode Group: None Status: Open Resolution: None Priority: 3 Submitted By: Finn Bock (bckfnn) Assigned to: M.-A. Lemburg (lemburg) Summary: repr diff between string and unicode. Initial Comment: A minor difference exists between the repr output of string and unicode. >>> "\x7f" '\x7f' >>> u"\x7f" u'⌂' >>> I can't think of any reason for this difference. ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 03:52 Message: Logged In: YES user_id=38388 I can't reproduce this with the current CVS version of Python on Linux. Which version did you use ? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 From noreply@sourceforge.net Wed Nov 28 12:34:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 04:34:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-485951 ] repr diff between string and unicode. Message-ID: Bugs item #485951, was opened at 2001-11-27 02:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 Category: Unicode Group: None Status: Open Resolution: None Priority: 3 Submitted By: Finn Bock (bckfnn) Assigned to: M.-A. Lemburg (lemburg) Summary: repr diff between string and unicode. Initial Comment: A minor difference exists between the repr output of string and unicode. >>> "\x7f" '\x7f' >>> u"\x7f" u'⌂' >>> I can't think of any reason for this difference. ---------------------------------------------------------------------- >Comment By: Finn Bock (bckfnn) Date: 2001-11-28 04:34 Message: Logged In: YES user_id=4201 I used 2.2b2. I had though the code in unicodeescape_string was at fault since it does: /* Map non-printable US ASCII to '\xhh' */ else if (ch < ' ' || ch >= 128) { *p++ = '\'; *p++ = 'x'; *p++ = hexdigit[(ch >> 4) & 0x000F]; *p++ = hexdigit[ch & 0x000F]; } where as string_repr does: else if (c < ' ' || c >= 0x7f) { sprintf(p, "\x%02x", c & 0xff); p += 4; The 0x7F character is treated differently. I can't compile C programs at the moment so I cant verify this on the CVS version. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 03:52 Message: Logged In: YES user_id=38388 I can't reproduce this with the current CVS version of Python on Linux. Which version did you use ? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 From noreply@sourceforge.net Wed Nov 28 12:56:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 04:56:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-485951 ] repr diff between string and unicode. Message-ID: Bugs item #485951, was opened at 2001-11-27 02:25 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 Category: Unicode Group: None >Status: Closed >Resolution: Fixed Priority: 3 Submitted By: Finn Bock (bckfnn) Assigned to: M.-A. Lemburg (lemburg) Summary: repr diff between string and unicode. Initial Comment: A minor difference exists between the repr output of string and unicode. >>> "\x7f" '\x7f' >>> u"\x7f" u'⌂' >>> I can't think of any reason for this difference. ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 04:56 Message: Logged In: YES user_id=38388 Ah, now I understand... I was misinterpreting the SF comment which probably got passed through some HTML-quoting function. You are right, \x7f should also be quoted. I'll fix that in CVS. ---------------------------------------------------------------------- Comment By: Finn Bock (bckfnn) Date: 2001-11-28 04:34 Message: Logged In: YES user_id=4201 I used 2.2b2. I had though the code in unicodeescape_string was at fault since it does: /* Map non-printable US ASCII to '\xhh' */ else if (ch < ' ' || ch >= 128) { *p++ = '\'; *p++ = 'x'; *p++ = hexdigit[(ch >> 4) & 0x000F]; *p++ = hexdigit[ch & 0x000F]; } where as string_repr does: else if (c < ' ' || c >= 0x7f) { sprintf(p, "\x%02x", c & 0xff); p += 4; The 0x7F character is treated differently. I can't compile C programs at the moment so I cant verify this on the CVS version. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 03:52 Message: Logged In: YES user_id=38388 I can't reproduce this with the current CVS version of Python on Linux. Which version did you use ? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485951&group_id=5470 From noreply@sourceforge.net Wed Nov 28 13:52:01 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 05:52:01 -0800 Subject: [Python-bugs-list] [ python-Bugs-484950 ] docs suggest no cyclic garbage collectio Message-ID: Bugs item #484950, was opened at 2001-11-23 10:50 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 Category: Documentation Group: None >Status: Open Resolution: None Priority: 5 Submitted By: Zooko Ozoko (zooko) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: docs suggest no cyclic garbage collectio Initial Comment: http://python.sourceforge.net/devel-docs/ext/refcounts.html Section 1.10 is a bit out of date -- it would lead some readers to believe that Python doesn't collect cyclical garbage. ---------------------------------------------------------------------- >Comment By: Zooko Ozoko (zooko) Date: 2001-11-28 05:52 Message: Logged In: YES user_id=52562 The last paragraph contrasts Python's chosen strategy, reference counting, with the more standard "automatic garbage collection". Most readers will know that reference counting suffers from the problem of not collecting cyclical garbage where garbage collection does not. Thus, seeing the two contrasted like this, and the reason for Python's lack of garbage collection being given as "there isn't a good one available", they may infer that Python doesn't collect cyclical garbage. I suggest to strike the last paragraph and replace it with something like: """ The strategy of reference counting traditionally suffers from the problem that the interpreter doesn't collect "cyclical garbage". Cyclical garbage is a set of objects which have been unlinked so that they are unreachable, but they contain references to each other so that each object's counter is non-zero. As of version 2.0, Python includes a "garbage cycle collector" which recovers the memory from cyclical garbage as well. (Note that there is an important caveat that the cycle collector does not free objects which contain `__del__' methods, although the normal non-cyclic reference counting system will free those objects. Please see XXX for more information.) """ Regards, Zooko ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-27 22:28 Message: Logged In: YES user_id=3066 Oops, that was *pending*, not closed! ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-27 22:19 Message: Logged In: YES user_id=3066 I must be missing something; what makes you think this implies cyclical garbage isn't collected? Marked pending awaiting further information. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=484950&group_id=5470 From noreply@sourceforge.net Wed Nov 28 13:56:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 05:56:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c Message-ID: Bugs item #486434, was opened at 2001-11-28 03:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: M.-A. Lemburg (lemburg) >Assigned to: M.-A. Lemburg (lemburg) Summary: Compiler complaints in posixmodule.c Initial Comment: The linker on Linux reports some warnings for posixmodule.c: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436: the use of `tempnam' is dangerous, better use `mkstemp' Perhaps we ought to follow the advice ?! ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 05:56 Message: Logged In: YES user_id=21627 No. The usage of tmpnam is not dangerous; we are just exposing it to the Python application. It may be reasonable to produce a warning if tmpnam is called. We cannot replace tempnam with mkstemp for the same reason Posix couldn't: one produces a string, the other one a file handle. What we could do is to expose mkstemp(3) where available. I don't see the value of that, though: it could be done only on systems where mkstemp is available, and we already expose tmpfile(3). In fact, the Linux man page for mkstemp(3) says # Don't use this function, use tmpfile(3) instead. It's # bet­ter defined and more portable. If you still think there is a need for action, please propose a patch. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 From noreply@sourceforge.net Wed Nov 28 14:17:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 06:17:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c Message-ID: Bugs item #486434, was opened at 2001-11-28 03:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 5 Submitted By: M.-A. Lemburg (lemburg) Assigned to: M.-A. Lemburg (lemburg) Summary: Compiler complaints in posixmodule.c Initial Comment: The linker on Linux reports some warnings for posixmodule.c: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436: the use of `tempnam' is dangerous, better use `mkstemp' Perhaps we ought to follow the advice ?! ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:17 Message: Logged In: YES user_id=38388 Hmm, the man page on SuSE Linux does not say anything about using tmpfile() instead of mkstemp(). BTW, the warnings are already in place. I wonder whether it wouldn't be better to remove the Python APIs for these functions altogether and instead provide interfaces for the mkstemp() and/or tempfile(). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 05:56 Message: Logged In: YES user_id=21627 No. The usage of tmpnam is not dangerous; we are just exposing it to the Python application. It may be reasonable to produce a warning if tmpnam is called. We cannot replace tempnam with mkstemp for the same reason Posix couldn't: one produces a string, the other one a file handle. What we could do is to expose mkstemp(3) where available. I don't see the value of that, though: it could be done only on systems where mkstemp is available, and we already expose tmpfile(3). In fact, the Linux man page for mkstemp(3) says # Don't use this function, use tmpfile(3) instead. It's # bet­ter defined and more portable. If you still think there is a need for action, please propose a patch. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 From noreply@sourceforge.net Wed Nov 28 14:24:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 06:24:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request >Status: Closed >Resolution: Works For Me Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: M.-A. Lemburg (lemburg) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:24 Message: Logged In: YES user_id=38388 BTW, you'll find a whole bunch of tools to "find" the Python installation in distutils. Closing the report since I don't see any point leaving it open ;-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-26 05:27 Message: Logged In: YES user_id=38388 Here's a summary of what I do to support the mx Extensions for multiple Python versions: Note: I use distutils on all platforms and distutils has builtin ways to find where the specific Python version I am targetting lives. On Unix: Since the only binary format distutils currently support is RPM, this is only relevant to the bdist_rpm command. Unfortunately, RPM does not inherit the environment, so you have to hard-code the path to the Python version into the RPM. It goes something like this: $(PYTHON) setup.py bdist_rpm \ --python $(PYTHON) \ --release $(PACKAGERELEASE) \ $(_RPM_REQUIRES) On Windows: Here, you have to run the setup.py file using the Python version you wish to build for, e.g. python2.0 setup.py bdist_win. I choose the Python version by hacking the PATH to point to the right version (this is needed to have sub-processes use the same Python version), e.g. wininst20.bat: @echo off call vcvars32 echo -------------------------------------------------------------- echo Building for Python 2.0... set PATH="d:\Python20;%PATH%" python setup.py clean --all python setup.py bdist_wininst -c -o That's about it. Hope it helps. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-23 18:16 Message: Logged In: YES user_id=31435 It clarifies the problem but mainly in the sense of clarifying that it *is* a problem: we don't require, or even recommend, any particular way of building extensions. IOW, "but I can't be certain" is correct, but not for a reason I can do anything about except to confirm that it's the truth. Marc-Andre Lemburg is our most prolific extension writer, so I'm reassigning this to him in the hope he'll explain what *he* does. You could do a lot worse than catering to whatever that is. ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 15:34 Message: Logged In: YES user_id=52572 The problem goes something like this: Assume I have a nice, generalized build system (I do, but not as nice as it will be soon ;-)), to which I want to add Python support. I want it to be possible to compile/test code against multiple Python versions, though that's not too relevant to the problem. I'd like finding the neccessary components (headers, libs, executables) to be relatively automatic, based on, e.g. the user's specification of a python base directory and a version number. On Unices it should be possible to do without the base directory unless python was configured --with-prefix=.... I have to assume that someone developing Python extension modules may either be using the source tree, or an installation specific to to the platform he's developing on. Someone who's just building an extension module is probably using a Python installation, but I can't be certain. So, does that clear things up a bit? -Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-15 15:15 Message: Logged In: YES user_id=31435 The source distro is just a snapshot of Python's CVS development tree, so it's a build tree, not an installation tree. "The std" Windows installation tree is the one created by the PythonLabs Windows installer. Similar Windows trees are created by the ActiveState and PythonWare Windows installers. "The std" Unix installation tree is whatever the heck "make install" does on Unix (and it's not the *build* tree either). Likewise for Macs, which rearrage the build tree in yet other ways for installation. The relationship between the build tree and the Windows installation tree is established by the 3000-line python20.wse script in PCbuild, which is input to the Wise installer-builder; I can't summarize it usefully here, and Windows installation also involves fiddling with registry entries. Afraid I'm still not clear on why you're asking, although I'm quite sure I'm not really helping . ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Wed Nov 28 14:25:44 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 06:25:44 -0800 Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c Message-ID: Bugs item #486434, was opened at 2001-11-28 03:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None >Priority: 3 Submitted By: M.-A. Lemburg (lemburg) Assigned to: M.-A. Lemburg (lemburg) Summary: Compiler complaints in posixmodule.c Initial Comment: The linker on Linux reports some warnings for posixmodule.c: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436: the use of `tempnam' is dangerous, better use `mkstemp' Perhaps we ought to follow the advice ?! ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:17 Message: Logged In: YES user_id=38388 Hmm, the man page on SuSE Linux does not say anything about using tmpfile() instead of mkstemp(). BTW, the warnings are already in place. I wonder whether it wouldn't be better to remove the Python APIs for these functions altogether and instead provide interfaces for the mkstemp() and/or tempfile(). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 05:56 Message: Logged In: YES user_id=21627 No. The usage of tmpnam is not dangerous; we are just exposing it to the Python application. It may be reasonable to produce a warning if tmpnam is called. We cannot replace tempnam with mkstemp for the same reason Posix couldn't: one produces a string, the other one a file handle. What we could do is to expose mkstemp(3) where available. I don't see the value of that, though: it could be done only on systems where mkstemp is available, and we already expose tmpfile(3). In fact, the Linux man page for mkstemp(3) says # Don't use this function, use tmpfile(3) instead. It's # bet­ter defined and more portable. If you still think there is a need for action, please propose a patch. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 From noreply@sourceforge.net Wed Nov 28 14:39:31 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 06:39:31 -0800 Subject: [Python-bugs-list] [ python-Bugs-485781 ] get_refer{ent,rer}s() doesn't use `is' Message-ID: Bugs item #485781, was opened at 2001-11-26 13:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 7 Submitted By: Zooko Ozoko (zooko) Assigned to: Martin v. Löwis (loewis) Summary: get_refer{ent,rer}s() doesn't use `is' Initial Comment: in Python 2.2b2 as packaged by debian, gc.get_referents() seems to be using equality testing rather than identity testing to find referents. Example: >>> l = [0,3,2,{},] >>> gc.get_referents({}) [Type help() for interactive help, or help(object) for help about object., [0, 3, 2, {}]] The list should not have appeared in the result. ---------------------------------------------------------------------- >Comment By: Zooko Ozoko (zooko) Date: 2001-11-28 06:39 Message: Logged In: YES user_id=52562 Yep! That patch fixes the problem with getting answers I didn't want *and* it fixes the seg faults! Now I'm making faster progress on diagnosing memory usage in Mojo Nation. Whoo-hoo! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-26 23:22 Message: Logged In: YES user_id=21627 Please try the attached patch (or report back if you cannot). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 From noreply@sourceforge.net Wed Nov 28 14:45:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 06:45:41 -0800 Subject: [Python-bugs-list] [ python-Bugs-486480 ] zipfile __del__ is broken Message-ID: Bugs item #486480, was opened at 2001-11-28 06:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486480&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Itamar Shtull-Trauring (itamar) Assigned to: Nobody/Anonymous (nobody) Summary: zipfile __del__ is broken Initial Comment: The __del__ method calls self.fp.close(), instead of calling self.close(). As a result, if you opened a zipfile in write mode and forgot to close() it, it will *not* be closed correctly when __del__ is called. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486480&group_id=5470 From noreply@sourceforge.net Wed Nov 28 15:32:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 07:32:41 -0800 Subject: [Python-bugs-list] [ python-Bugs-486500 ] cgitb,inspect,pydoc Message-ID: Bugs item #486500, was opened at 2001-11-28 07:32 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486500&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: cgitb,inspect,pydoc Initial Comment: the actual Beta-Version 2.2b2 of Python does not contain any documentation for the above mentioned Modules. So, the one who looks for this can't find it via /doc/ TXN lh, bs ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486500&group_id=5470 From noreply@sourceforge.net Wed Nov 28 15:52:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 07:52:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-485781 ] get_refer{ent,rer}s() doesn't use `is' Message-ID: Bugs item #485781, was opened at 2001-11-26 13:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 7 Submitted By: Zooko Ozoko (zooko) Assigned to: Martin v. Löwis (loewis) Summary: get_refer{ent,rer}s() doesn't use `is' Initial Comment: in Python 2.2b2 as packaged by debian, gc.get_referents() seems to be using equality testing rather than identity testing to find referents. Example: >>> l = [0,3,2,{},] >>> gc.get_referents({}) [Type help() for interactive help, or help(object) for help about object., [0, 3, 2, {}]] The list should not have appeared in the result. ---------------------------------------------------------------------- >Comment By: Zooko Ozoko (zooko) Date: 2001-11-28 07:52 Message: Logged In: YES user_id=52562 Whoops, although it survived an overnight run of continuously doing `get_referrers({})' (which formerly would have triggered a seg fault), it did not survive the following interactive code: >>> for ref in refs[:]: ... for k, v in locals().items(): ... if (ref is k) or (ref is v): ... refs.remove(ref) ... That got a segfault. `refs' was a list of objects (namely, all objects in the current runtime that held a reference to an empty dict. ---------------------------------------------------------------------- Comment By: Zooko Ozoko (zooko) Date: 2001-11-28 06:39 Message: Logged In: YES user_id=52562 Yep! That patch fixes the problem with getting answers I didn't want *and* it fixes the seg faults! Now I'm making faster progress on diagnosing memory usage in Mojo Nation. Whoo-hoo! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-26 23:22 Message: Logged In: YES user_id=21627 Please try the attached patch (or report back if you cannot). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 From noreply@sourceforge.net Wed Nov 28 15:53:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 07:53:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-486480 ] zipfile __del__ is broken Message-ID: Bugs item #486480, was opened at 2001-11-28 06:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486480&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Itamar Shtull-Trauring (itamar) Assigned to: Nobody/Anonymous (nobody) Summary: zipfile __del__ is broken Initial Comment: The __del__ method calls self.fp.close(), instead of calling self.close(). As a result, if you opened a zipfile in write mode and forgot to close() it, it will *not* be closed correctly when __del__ is called. ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-28 07:53 Message: Logged In: YES user_id=31392 I guess it's a bug since __del__() doesn't do what the doc string says, but shouldn't the program be calling close() explicitly. It makes some (a little) sense to think of __del__() only as a stop-gap measure to prevent a program from leaking file descriptors. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486480&group_id=5470 From noreply@sourceforge.net Wed Nov 28 15:56:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 07:56:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-485679 ] 2.2b2 Won't build on Solaris 2.8 Message-ID: Bugs item #485679, was opened at 2001-11-26 08:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485679&group_id=5470 Category: Build Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: David M. Beazley (beazley) >Assigned to: Martin v. Löwis (loewis) Summary: 2.2b2 Won't build on Solaris 2.8 Initial Comment: 2.2b2 doesn't build on Sparc Solaris 2.8 with gcc-2.95.2 and gmake-3.77. The following error occurs: make: *** No rule to make target `Python/thread_*.h', needed by `Python/thread.o'. Stop. This occurs with the standard build process (./configure; make). If I had to take a wild guess on this, perhaps the build process (configure or make) is using a non-portable feature of the shell and/or make. -- Dave ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-28 07:55 Message: Logged In: YES user_id=31392 Can you take a look Martin? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485679&group_id=5470 From noreply@sourceforge.net Wed Nov 28 16:22:07 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 08:22:07 -0800 Subject: [Python-bugs-list] [ python-Bugs-476852 ] Some bad macros in abstract.h Message-ID: Bugs item #476852, was opened at 2001-10-31 10:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476852&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Jeremy Hylton (jhylton) Summary: Some bad macros in abstract.h Initial Comment: Greg Chapman reported in c.l.py: abstract.h defines the following: #define PyMapping_DelItemString(O,K) PyDict_DelItemString((O),(K)) #define PyMapping_DelItem(O,K) PyDict_DelItem((O),(K)) It seems to me they should delegate to PyObject_DelItem (and PyObject_DelItemString, which will have to be added). ---------------------------------------------------------------------- >Comment By: Jeremy Hylton (jhylton) Date: 2001-11-28 08:22 Message: Logged In: YES user_id=31392 Fixed in rev 2.42 of abstract.h ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=476852&group_id=5470 From noreply@sourceforge.net Wed Nov 28 17:03:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 09:03:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-486527 ] xmlrpclib can produce ill-formed XML Message-ID: Bugs item #486527, was opened at 2001-11-28 09:03 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: A.M. Kuchling (akuchling) Assigned to: Nobody/Anonymous (nobody) Summary: xmlrpclib can produce ill-formed XML Initial Comment: I discovered this when a control character got into a string. import xmlrpclib data = xmlrpclib.dumps( (chr(17),) ) print data The resulting XML output isn't well-formed, because chr(17) isn't a legal character. The fix is conceptually simple -- just check for illegal characters in the dump_string() method -- but I'm not sure how to do this check efficiently. Fredrik, would it be OK with you to use a regex? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 From noreply@sourceforge.net Wed Nov 28 17:03:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 09:03:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-486527 ] xmlrpclib can produce ill-formed XML Message-ID: Bugs item #486527, was opened at 2001-11-28 09:03 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 5 Submitted By: A.M. Kuchling (akuchling) >Assigned to: Fredrik Lundh (effbot) Summary: xmlrpclib can produce ill-formed XML Initial Comment: I discovered this when a control character got into a string. import xmlrpclib data = xmlrpclib.dumps( (chr(17),) ) print data The resulting XML output isn't well-formed, because chr(17) isn't a legal character. The fix is conceptually simple -- just check for illegal characters in the dump_string() method -- but I'm not sure how to do this check efficiently. Fredrik, would it be OK with you to use a regex? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 From noreply@sourceforge.net Wed Nov 28 17:08:09 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 09:08:09 -0800 Subject: [Python-bugs-list] [ python-Bugs-486530 ] replace sprintf with PyOS_snprintf Message-ID: Bugs item #486530, was opened at 2001-11-28 09:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486530&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 8 Submitted By: Jeremy Hylton (jhylton) Assigned to: Nobody/Anonymous (nobody) Summary: replace sprintf with PyOS_snprintf Initial Comment: Some or all of the sprintf calls we make are vulnerable to buffer overflows. A few of these calls use stack-allocated buffers, which are real security problems. MAL has fixed three of them, but if we're going to fix any we need to fix them all. We'll try to finish this task as soon as possible. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486530&group_id=5470 From noreply@sourceforge.net Wed Nov 28 17:35:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 09:35:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-485766 ] install hangs Message-ID: Bugs item #485766, was opened at 2001-11-26 12:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485766&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: install hangs Initial Comment: Install on NT 4.0 appears to go fine until the final step. It says it is finished, but clicking on OK (or finish) leaves server with a blue screen (python install, not system crash). System just sits there & does not change.. CPU at 30% or less...mostly less. Help! User is frantic! Thanks! ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-28 09:35 Message: Logged In: YES user_id=31435 Will close this at the end of the week if there's no followup from Anonymous. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-26 16:33 Message: Logged In: YES user_id=31435 Log in to an Administrator account. Don't install over a network (install from local copy). Accept defaults. Ensure there's enough disk space. Kill virus checkers (etc) for the duration ... this is all "general flaky Windows" advice, there's nothing specific to Python here. If all else fails, try the ActiveState or Secret Labs installer. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485766&group_id=5470 From noreply@sourceforge.net Wed Nov 28 18:22:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 10:22:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-486565 ] Mac OS 10.1: unobvious: --with-suffix Message-ID: Bugs item #486565, was opened at 2001-11-28 10:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486565&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Jack Jansen (jackjansen) Summary: Mac OS 10.1: unobvious: --with-suffix Initial Comment: First attempts to install Python-2.2b2 under Mac OS 10.1 on a HFS+ filesystem failed at the point of linking the python executable. When browsing comp.lang.python it became clear that the --with- suffix=.exe configure option has to be used. Currently, configure only reports: checking for executable suffix... no checking for --with-suffix... It would be a big improvement if configure could issue a very visible warning, or even better, supply the suffix automatically and tell the user at the very end that python is python.exe. Thanks! Ralf ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486565&group_id=5470 From noreply@sourceforge.net Wed Nov 28 18:22:37 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 10:22:37 -0800 Subject: [Python-bugs-list] [ python-Bugs-486527 ] xmlrpclib can produce ill-formed XML Message-ID: Bugs item #486527, was opened at 2001-11-28 09:03 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None >Priority: 3 Submitted By: A.M. Kuchling (akuchling) Assigned to: Fredrik Lundh (effbot) Summary: xmlrpclib can produce ill-formed XML Initial Comment: I discovered this when a control character got into a string. import xmlrpclib data = xmlrpclib.dumps( (chr(17),) ) print data The resulting XML output isn't well-formed, because chr(17) isn't a legal character. The fix is conceptually simple -- just check for illegal characters in the dump_string() method -- but I'm not sure how to do this check efficiently. Fredrik, would it be OK with you to use a regex? ---------------------------------------------------------------------- >Comment By: Fredrik Lundh (effbot) Date: 2001-11-28 10:22 Message: Logged In: YES user_id=38376 I ignored this on purpose: instead of forcing everyone to pay a (rather big) performance penalty, the application must make sure to use the right data type. The documentation probably needs fixing, though... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 From noreply@sourceforge.net Wed Nov 28 19:10:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 11:10:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-486527 ] xmlrpclib can produce ill-formed XML Message-ID: Bugs item #486527, was opened at 2001-11-28 09:03 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 3 Submitted By: A.M. Kuchling (akuchling) Assigned to: Fredrik Lundh (effbot) Summary: xmlrpclib can produce ill-formed XML Initial Comment: I discovered this when a control character got into a string. import xmlrpclib data = xmlrpclib.dumps( (chr(17),) ) print data The resulting XML output isn't well-formed, because chr(17) isn't a legal character. The fix is conceptually simple -- just check for illegal characters in the dump_string() method -- but I'm not sure how to do this check efficiently. Fredrik, would it be OK with you to use a regex? ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 11:10 Message: Logged In: YES user_id=38388 I suppose a xmlescape codec would be a good addition to the set of codecs in Python. Maybe for 2.3... ---------------------------------------------------------------------- Comment By: Fredrik Lundh (effbot) Date: 2001-11-28 10:22 Message: Logged In: YES user_id=38376 I ignored this on purpose: instead of forcing everyone to pay a (rather big) performance penalty, the application must make sure to use the right data type. The documentation probably needs fixing, though... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 From noreply@sourceforge.net Wed Nov 28 19:31:05 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 11:31:05 -0800 Subject: [Python-bugs-list] [ python-Bugs-486527 ] xmlrpclib can produce ill-formed XML Message-ID: Bugs item #486527, was opened at 2001-11-28 09:03 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 Category: Python Library Group: None Status: Open Resolution: None Priority: 3 Submitted By: A.M. Kuchling (akuchling) >Assigned to: A.M. Kuchling (akuchling) Summary: xmlrpclib can produce ill-formed XML Initial Comment: I discovered this when a control character got into a string. import xmlrpclib data = xmlrpclib.dumps( (chr(17),) ) print data The resulting XML output isn't well-formed, because chr(17) isn't a legal character. The fix is conceptually simple -- just check for illegal characters in the dump_string() method -- but I'm not sure how to do this check efficiently. Fredrik, would it be OK with you to use a regex? ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2001-11-28 11:31 Message: Logged In: YES user_id=11375 An xmlescape codec wouldn't help with this; it could turn chr(17) into , but that character is still illegal. Fixing the docs seems the most reasonable thing to do; I'll try to contribute a documentation patch. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 11:10 Message: Logged In: YES user_id=38388 I suppose a xmlescape codec would be a good addition to the set of codecs in Python. Maybe for 2.3... ---------------------------------------------------------------------- Comment By: Fredrik Lundh (effbot) Date: 2001-11-28 10:22 Message: Logged In: YES user_id=38376 I ignored this on purpose: instead of forcing everyone to pay a (rather big) performance penalty, the application must make sure to use the right data type. The documentation probably needs fixing, though... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486527&group_id=5470 From noreply@sourceforge.net Wed Nov 28 20:21:14 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 12:21:14 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-28 12:21 Message: Logged In: YES user_id=85984 Attached are three files: - pyconfig.h from a fresh gcc 2.7.2.1 build - config.log from a fresh gcc 2.7.2.1 build - /usr/include/netdb.h - the getaddrinfo manual page from my system Indeed, EAI_ADDRFAMILY is defined in /usr/include/netdb.h, but EAI_MAX is not. Let me know if I can provide any more information. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 01:30 Message: Logged In: YES user_id=21627 Ok, then we need to investigate the configure results. Can you please attach pyconfig.h and config.log to this report? Also, can you please confirm that you build Python from scratch with 2.7.2.1 (i.e untarring a fresh source tree, to avoid using a config.cache left over from gcc 2.95.2)? Furthermore, can you please report details of the getaddrinfo implementation on your system? Does it provide one? In the C library? If not, can you confirm that EAI_ADDRFAMILY is defined in your system headers, but EAI_MAX is not? If yes, can you speculate as to why configure thinks your system does not have getaddrinfo? ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-27 15:21 Message: Logged In: YES user_id=85984 I probably should have included the output from gcc 2.7.2.1 instead which I know is properly installed. The results are similar, although there is no mention of `wint_t': % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from Include/pyport.h:93, from Include/Python.h:60, from ./Modules/socketmodule.c:78: /usr/include/math.h:177: warning: function declaration isn't a prototype /usr/include/math.h:257: warning: function declaration isn't a prototype In file included from Include/unicodeobject.h:118, from Include/Python.h:69, from ./Modules/socketmodule.c:78: /usr/include/wchar.h:15: warning: function declaration isn't a prototype In file included from /usr/include/signal.h:46, from ./Modules/socketmodule.c:140: /usr/include/sys/signal.h:121: warning: function declaration isn't a prototype /usr/include/sys/signal.h:164: warning: function declaration isn't a prototype Modules/getaddrinfo.c: In function `gai_strerror': In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Wed Nov 28 20:42:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 12:42:41 -0800 Subject: [Python-bugs-list] [ python-Bugs-486565 ] Mac OS 10.1: unobvious: --with-suffix Message-ID: Bugs item #486565, was opened at 2001-11-28 10:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486565&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Jack Jansen (jackjansen) Summary: Mac OS 10.1: unobvious: --with-suffix Initial Comment: First attempts to install Python-2.2b2 under Mac OS 10.1 on a HFS+ filesystem failed at the point of linking the python executable. When browsing comp.lang.python it became clear that the --with- suffix=.exe configure option has to be used. Currently, configure only reports: checking for executable suffix... no checking for --with-suffix... It would be a big improvement if configure could issue a very visible warning, or even better, supply the suffix automatically and tell the user at the very end that python is python.exe. Thanks! Ralf ---------------------------------------------------------------------- >Comment By: Jack Jansen (jackjansen) Date: 2001-11-28 12:42 Message: Logged In: YES user_id=45365 Yes, this is a serious problem. Configure _does_ give a warning, but it gets lost in the stream of output it produces. The problem is that I don't know how to test whether we're building on an HFS+ filesystem. Only then should we add the --with-suffix automatically, if you're building on a UFS or NFS filesystem there is no problem. Would you happen to know of a way to test for this? Or should I simply always use --with-suffix on OSX? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486565&group_id=5470 From noreply@sourceforge.net Wed Nov 28 20:58:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 12:58:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-486278 ] SystemError: Python/getargs.c:1086: bad Message-ID: Bugs item #486278, was opened at 2001-11-27 17:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 Category: Extension Modules Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: SystemError: Python/getargs.c:1086: bad Initial Comment: This looks like a new problem introduced in 2.2b2 (or something has changed in the way we are to write extention modules in C)...I'm using Solaris 2.6 with no configure options other then --prefix=... I started with two fresh builds and copied the same files their proper places. One was Python-2.2b1 and the other was Python-2.2b2. My extention module is fairly a fairly simple conversion utility. Both builds complete cleanly. Here is the output from 2.2b1: ----------------------- Python 2.2b1 (#1, Nov 27 2001, 19:28:20) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import Cstimbase >>> Cstimbase.hex2wire('aaff11') '\xaa\xff\x11' >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) 'aaff11' >>> --------------------------- Here is the exact same module running in 2.2b2: ------------------------------- Python 2.2b2 (#1, Nov 27 2001, 16:42:01) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import Cstimbase >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) Traceback (most recent call last): File "", line 1, in ? SystemError: Python/getargs.c:1086: bad argument to internal function >>> ----------------------------- So what changed in the way you write C extention modules? Thanks, Sam (stannous@employees.org) ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-28 12:58 Message: Logged In: NO You're correct...I fixed my code and it's fine now. (when 2.2 final is released, a blurb in the CHANGES file that states that new sanity checks are being done would be nice....either that or a slightly more meaningful error message ;-) Please close this bug. Thanks again, Sam ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 19:49 Message: Logged In: YES user_id=31435 Do you call PyArg_ParseTupleAndKeywords() in your extension? The error you're seeing is a new sanity check, verifying that the number of keyword args passed in kwlist is no larger than the number of argument codes passed in the format string. It didn't used to check this, and it was possible for Python to dump core as a result. Please check all your uses of ParseTupleAndKeywords() -- I can't do it for you without seeing your code. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 From noreply@sourceforge.net Wed Nov 28 22:01:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 14:01:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-486565 ] Mac OS 10.1: unobvious: --with-suffix Message-ID: Bugs item #486565, was opened at 2001-11-28 10:21 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486565&group_id=5470 Category: Macintosh Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Ralf W. Grosse-Kunstleve (rwgk) Assigned to: Jack Jansen (jackjansen) Summary: Mac OS 10.1: unobvious: --with-suffix Initial Comment: First attempts to install Python-2.2b2 under Mac OS 10.1 on a HFS+ filesystem failed at the point of linking the python executable. When browsing comp.lang.python it became clear that the --with- suffix=.exe configure option has to be used. Currently, configure only reports: checking for executable suffix... no checking for --with-suffix... It would be a big improvement if configure could issue a very visible warning, or even better, supply the suffix automatically and tell the user at the very end that python is python.exe. Thanks! Ralf ---------------------------------------------------------------------- >Comment By: Ralf W. Grosse-Kunstleve (rwgk) Date: 2001-11-28 14:01 Message: Logged In: YES user_id=71407 > Yes, this is a serious problem. Configure _does_ give a warning, ... I cannot even see the warning when I am looking for it. A full output of running 2.2b2 configure under MacOS 10.1 is available at this location: http://cci.lbl.gov/~rwgk/tmp/MacOS10.1_Python- 2.2b2_configure_output.txt Is the warning in there? > Would you happen to know of a way to test for this? You could simply try to detect the very feature that is causing the default build to fail: 1. Create an empty file with some dummy, lowercase name: touch zap 2. Now test the existence of the file under the uppercase name: test -f ZAP 3. On HFS, $status == 0, on UFS or NFS, $status == 1. Ralf ---------------------------------------------------------------------- Comment By: Jack Jansen (jackjansen) Date: 2001-11-28 12:42 Message: Logged In: YES user_id=45365 Yes, this is a serious problem. Configure _does_ give a warning, but it gets lost in the stream of output it produces. The problem is that I don't know how to test whether we're building on an HFS+ filesystem. Only then should we add the --with-suffix automatically, if you're building on a UFS or NFS filesystem there is no problem. Would you happen to know of a way to test for this? Or should I simply always use --with-suffix on OSX? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486565&group_id=5470 From noreply@sourceforge.net Wed Nov 28 22:12:52 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 14:12:52 -0800 Subject: [Python-bugs-list] [ python-Bugs-486713 ] 2.2b2 IDLE on W2k does not load Message-ID: Bugs item #486713, was opened at 2001-11-28 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486713&group_id=5470 Category: IDLE Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Guido van Rossum (gvanrossum) Summary: 2.2b2 IDLE on W2k does not load Initial Comment: Hi, 2.2b2 IDLE on W2k does not load, I uninstalled the previous version(which worked fine), installed 2.2b2 which seemed to go Ok, when I now load IDLE, the hourglass appears for about 2 seconds, then nothing. I am adminstrator on W2k Pro. The command line works OK. Thanks;....Andrew ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486713&group_id=5470 From noreply@sourceforge.net Wed Nov 28 22:16:13 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 14:16:13 -0800 Subject: [Python-bugs-list] [ python-Bugs-486713 ] 2.2b2 IDLE on W2k does not load Message-ID: Bugs item #486713, was opened at 2001-11-28 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486713&group_id=5470 Category: IDLE Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) >Assigned to: Tim Peters (tim_one) Summary: 2.2b2 IDLE on W2k does not load Initial Comment: Hi, 2.2b2 IDLE on W2k does not load, I uninstalled the previous version(which worked fine), installed 2.2b2 which seemed to go Ok, when I now load IDLE, the hourglass appears for about 2 seconds, then nothing. I am adminstrator on W2k Pro. The command line works OK. Thanks;....Andrew ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486713&group_id=5470 From noreply@sourceforge.net Wed Nov 28 23:00:56 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 15:00:56 -0800 Subject: [Python-bugs-list] [ python-Bugs-486713 ] 2.2b2 IDLE on W2k does not load Message-ID: Bugs item #486713, was opened at 2001-11-28 14:12 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486713&group_id=5470 Category: IDLE Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: 2.2b2 IDLE on W2k does not load Initial Comment: Hi, 2.2b2 IDLE on W2k does not load, I uninstalled the previous version(which worked fine), installed 2.2b2 which seemed to go Ok, when I now load IDLE, the hourglass appears for about 2 seconds, then nothing. I am adminstrator on W2k Pro. The command line works OK. Thanks;....Andrew ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-28 15:00 Message: Logged In: YES user_id=31435 Works for me, and there are no other reports of problems here. Needs more info. Try running Tools\idle\idle.pyw from a DOS box, using python.exe, and see whether that displays any error msgs. The Start menu shortcut uses pythonw.exe (note the trailing "w"), and error msgs end up in the bit bucket then. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486713&group_id=5470 From noreply@sourceforge.net Wed Nov 28 23:17:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 15:17:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-486480 ] zipfile __del__ is broken Message-ID: Bugs item #486480, was opened at 2001-11-28 06:45 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486480&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Itamar Shtull-Trauring (itamar) >Assigned to: Tim Peters (tim_one) Summary: zipfile __del__ is broken Initial Comment: The __del__ method calls self.fp.close(), instead of calling self.close(). As a result, if you opened a zipfile in write mode and forgot to close() it, it will *not* be closed correctly when __del__ is called. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-28 15:17 Message: Logged In: YES user_id=31435 Fixed, in Lib/zipfile.py; new revision: 1.19 ---------------------------------------------------------------------- Comment By: Jeremy Hylton (jhylton) Date: 2001-11-28 07:53 Message: Logged In: YES user_id=31392 I guess it's a bug since __del__() doesn't do what the doc string says, but shouldn't the program be calling close() explicitly. It makes some (a little) sense to think of __del__() only as a stop-gap measure to prevent a program from leaking file descriptors. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486480&group_id=5470 From noreply@sourceforge.net Thu Nov 29 02:50:30 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 18:50:30 -0800 Subject: [Python-bugs-list] [ python-Bugs-477023 ] pdb: unexpected path confuses Emacs Message-ID: Bugs item #477023, was opened at 2001-10-31 16:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477023&group_id=5470 Category: Python Library Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Jonathan Mark (jhmark) >Assigned to: Guido van Rossum (gvanrossum) Summary: pdb: unexpected path confuses Emacs Initial Comment: With python 2.2b1, Start the debugger like this on any script. Notice the stack frame display in which has a path name prepended to it. This confuses the emacs Python debugging mode (python-mode.el) causing it not to recognize the line as a stack frame, and thus not to display the (Pdb) prompt for the user. $ ./bin/python ./lib/python2.2/pdb.py ~/tmp/foo.py > /hda3/sys/Python-2.2b1-local/(0)?() (Pdb) in the same scenario, python 1.5 does not confuse emacs because it displays the line like this: > (0)?() ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-28 18:50 Message: Logged In: YES user_id=6380 Fixed in CVS now. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-10-31 19:53 Message: Logged In: YES user_id=6380 Here's a patch to bdb.py that might fix this. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=477023&group_id=5470 From noreply@sourceforge.net Thu Nov 29 03:30:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 19:30:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-486278 ] SystemError: Python/getargs.c:1086: bad Message-ID: Bugs item #486278, was opened at 2001-11-27 17:13 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 Category: Extension Modules Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Tim Peters (tim_one) Summary: SystemError: Python/getargs.c:1086: bad Initial Comment: This looks like a new problem introduced in 2.2b2 (or something has changed in the way we are to write extention modules in C)...I'm using Solaris 2.6 with no configure options other then --prefix=... I started with two fresh builds and copied the same files their proper places. One was Python-2.2b1 and the other was Python-2.2b2. My extention module is fairly a fairly simple conversion utility. Both builds complete cleanly. Here is the output from 2.2b1: ----------------------- Python 2.2b1 (#1, Nov 27 2001, 19:28:20) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> >>> import Cstimbase >>> Cstimbase.hex2wire('aaff11') '\xaa\xff\x11' >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) 'aaff11' >>> --------------------------- Here is the exact same module running in 2.2b2: ------------------------------- Python 2.2b2 (#1, Nov 27 2001, 16:42:01) [GCC 2.9-cisco-98r1] on sunos5 Type "help", "copyright", "credits" or "license" for more information. >>> import Cstimbase >>> x = Cstimbase.hex2wire('aaff11') >>> Cstimbase.wire2hex(x) Traceback (most recent call last): File "", line 1, in ? SystemError: Python/getargs.c:1086: bad argument to internal function >>> ----------------------------- So what changed in the way you write C extention modules? Thanks, Sam (stannous@employees.org) ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-28 19:30 Message: Logged In: YES user_id=31435 Good points! Misc/NEWS; new revision: 1.321 Python/getargs.c; new revision: 2.89 point out the new checks (they're actually repaired checks - - the old checks simply didn't work), change the exceptions raised from SystemError to RuntimeError, and add more- informative msgs. The msg in this particular case will be "more keyword list entries than argument specifiers". ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-28 12:58 Message: Logged In: NO You're correct...I fixed my code and it's fine now. (when 2.2 final is released, a blurb in the CHANGES file that states that new sanity checks are being done would be nice....either that or a slightly more meaningful error message ;-) Please close this bug. Thanks again, Sam ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 19:49 Message: Logged In: YES user_id=31435 Do you call PyArg_ParseTupleAndKeywords() in your extension? The error you're seeing is a new sanity check, verifying that the number of keyword args passed in kwlist is no larger than the number of argument codes passed in the format string. It didn't used to check this, and it was possible for Python to dump core as a result. Please check all your uses of ParseTupleAndKeywords() -- I can't do it for you without seeing your code. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486278&group_id=5470 From noreply@sourceforge.net Thu Nov 29 04:56:12 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 20:56:12 -0800 Subject: [Python-bugs-list] [ python-Bugs-486826 ] bogus error message on bad parameters Message-ID: Bugs item #486826, was opened at 2001-11-28 20:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bogus error message on bad parameters Initial Comment: I was writing a CGI and ran across the following error message: TypeError: 'in ' requires character as left operand, which drove me batty as I keep thinking it was the left operand when in reality it was the right operand.... I've simplified the problem to Python 2.1.1 (#1, Oct 30 2001, 09:41:48) [GCC 2.95.2 19991024 (release)] on darwin1 Type "copyright", "credits" or "license" for more information. >>> mystrings = "AE,Developer" >>> "AE" in mystrings Traceback (most recent call last): File "", line 1, in ? TypeError: 'in ' requires character as left operand >>> maybe it's just me but I keep thinking that "AE" is the left operand and it has a character.... ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 From noreply@sourceforge.net Thu Nov 29 05:00:42 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 21:00:42 -0800 Subject: [Python-bugs-list] [ python-Bugs-479967 ] Python/C API manual has no index section Message-ID: Bugs item #479967, was opened at 2001-11-09 04:41 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 7 Submitted By: Thomas Heller (theller) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Python/C API manual has no index section Initial Comment: In the development docs, the Python/C API manual the index section is missing. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-28 21:00 Message: Logged In: YES user_id=3066 Yes -- this is the same bug (I hope!). The "Extending & Embedding" manual's index is also missing. ---------------------------------------------------------------------- Comment By: Greg Chapman (glchapman) Date: 2001-11-19 06:18 Message: Logged In: YES user_id=86307 The docs included with the Windows 2.2b2 distribution only have indexes (genindex.html) for the lib and mac subdirectories. The indexes for the C API and Python Reference Manual are missing (I assume this is the same bug as reported here). ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-09 06:43 Message: Logged In: YES user_id=3066 Raised the severity of this -- this *must* be fixed! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=479967&group_id=5470 From noreply@sourceforge.net Thu Nov 29 05:06:46 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 21:06:46 -0800 Subject: [Python-bugs-list] [ python-Bugs-485165 ] PyEval_CallObject not in the index Message-ID: Bugs item #485165, was opened at 2001-11-24 12:08 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485165&group_id=5470 Category: Documentation Group: Python 2.1.1 >Status: Closed Resolution: None Priority: 5 Submitted By: Cl. Schmidt (clemm) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: PyEval_CallObject not in the index Initial Comment: In the Python/C API Reference Manual, the function PyEval_CallObject is missing in the index. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-28 21:06 Message: Logged In: YES user_id=3066 There's no index entry because PyEval_CallObject() is not documented (yet). This falls into the category of the general incompleteness of the document, which we are aware of and are working on (albiet slowly). I have added a index entry for this function in the "Extending & Embedding" manual; see the section "Calling Python Functions from C" for some general discussion of this function. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485165&group_id=5470 From noreply@sourceforge.net Thu Nov 29 05:44:18 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Wed, 28 Nov 2001 21:44:18 -0800 Subject: [Python-bugs-list] [ python-Bugs-485678 ] Use of __del__ on descriptors ambiguous Message-ID: Bugs item #485678, was opened at 2001-11-26 08:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485678&group_id=5470 Category: Type/class unification Group: Python 2.2 Status: Open Resolution: None Priority: 7 Submitted By: Guido van Rossum (gvanrossum) Assigned to: Guido van Rossum (gvanrossum) Summary: Use of __del__ on descriptors ambiguous Initial Comment: When deleting an attribute described by a descriptor implemented in Python, the descriptor's __del__ method is called by the slot_tp_descr_set dispatch function. This is bogus -- __del__ already has a different meaning. This should be renamed to __delete__. (XXX Should we also rename __get__ and __set__ to something more descriptive, e.g. __descr_get__ and and __descr_set__? Or maybe we should use __prop_ as a prefix?) ---------------------------------------------------------------------- >Comment By: Guido van Rossum (gvanrossum) Date: 2001-11-28 21:44 Message: Logged In: YES user_id=6380 Here's a patch that renames the method to __delete__. Anyone got problems with checking this in? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485678&group_id=5470 From noreply@sourceforge.net Thu Nov 29 18:07:38 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 10:07:38 -0800 Subject: [Python-bugs-list] [ python-Bugs-487147 ] Differ Object doc typo Message-ID: Bugs item #487147, was opened at 2001-11-29 10:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487147&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Differ Object doc typo Initial Comment: In lib docs, section 4.4.3 end: "The delta generated also consists of newline- terminated strings, ready to be printed as-is via the writeline() method of a file-like object. " When writelines() method was intended. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487147&group_id=5470 From noreply@sourceforge.net Thu Nov 29 18:14:41 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 10:14:41 -0800 Subject: [Python-bugs-list] [ python-Bugs-485781 ] get_refer{ent,rer}s() doesn't use `is' Message-ID: Bugs item #485781, was opened at 2001-11-26 13:16 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Fixed Priority: 7 Submitted By: Zooko Ozoko (zooko) Assigned to: Martin v. Löwis (loewis) Summary: get_refer{ent,rer}s() doesn't use `is' Initial Comment: in Python 2.2b2 as packaged by debian, gc.get_referents() seems to be using equality testing rather than identity testing to find referents. Example: >>> l = [0,3,2,{},] >>> gc.get_referents({}) [Type help() for interactive help, or help(object) for help about object., [0, 3, 2, {}]] The list should not have appeared in the result. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 10:14 Message: Logged In: YES user_id=21627 The patch has been committed as gcmodule.c 2.31. Since it fixes *this* bug (about using ==), I'll close this report. As for the problem you are still seeing: It would be good if the segfault could be analysed in a debugger (using the old core file if you cannot reproduce it in a life debugger). I have the suspicion that this is a different problem: refs.remove will also use ==, not is. That still shouldn't cause a segfault, but it appears to be a different bug. If you have further details, don't hesitate to open a new bug report. ---------------------------------------------------------------------- Comment By: Zooko Ozoko (zooko) Date: 2001-11-28 07:52 Message: Logged In: YES user_id=52562 Whoops, although it survived an overnight run of continuously doing `get_referrers({})' (which formerly would have triggered a seg fault), it did not survive the following interactive code: >>> for ref in refs[:]: ... for k, v in locals().items(): ... if (ref is k) or (ref is v): ... refs.remove(ref) ... That got a segfault. `refs' was a list of objects (namely, all objects in the current runtime that held a reference to an empty dict. ---------------------------------------------------------------------- Comment By: Zooko Ozoko (zooko) Date: 2001-11-28 06:39 Message: Logged In: YES user_id=52562 Yep! That patch fixes the problem with getting answers I didn't want *and* it fixes the seg faults! Now I'm making faster progress on diagnosing memory usage in Mojo Nation. Whoo-hoo! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-26 23:22 Message: Logged In: YES user_id=21627 Please try the attached patch (or report back if you cannot). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=485781&group_id=5470 From noreply@sourceforge.net Thu Nov 29 18:21:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 10:21:33 -0800 Subject: [Python-bugs-list] [ python-Bugs-487152 ] Framework reports wrong sys.executable Message-ID: Bugs item #487152, was opened at 2001-11-29 10:20 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487152&group_id=5470 Category: Macintosh Group: Platform-specific Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jack Jansen (jackjansen) Summary: Framework reports wrong sys.executable Initial Comment: I've run in to this a couple of times since I have been using Python exclusively on OS X built as a Framework. When built and installed as a framework, python reports that sys.executable is: /Library/Frameworks/Python.framework/Versions/ 2.2/Python Which may or may not be technically right. But since certain scripts assume they can execute the value of sys.executable to get a new python interpreter, they fail. Instead, they need to be pointed to /Library/Frameworks/Python.framework/Versions/ 2.2/bin/python.exe Not sure if and how this can be fixed, but it would be nice. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487152&group_id=5470 From noreply@sourceforge.net Thu Nov 29 18:23:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 10:23:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-486826 ] bogus error message on bad parameters Message-ID: Bugs item #486826, was opened at 2001-11-28 20:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bogus error message on bad parameters Initial Comment: I was writing a CGI and ran across the following error message: TypeError: 'in ' requires character as left operand, which drove me batty as I keep thinking it was the left operand when in reality it was the right operand.... I've simplified the problem to Python 2.1.1 (#1, Oct 30 2001, 09:41:48) [GCC 2.95.2 19991024 (release)] on darwin1 Type "copyright", "credits" or "license" for more information. >>> mystrings = "AE,Developer" >>> "AE" in mystrings Traceback (most recent call last): File "", line 1, in ? TypeError: 'in ' requires character as left operand >>> maybe it's just me but I keep thinking that "AE" is the left operand and it has a character.... ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 10:23 Message: Logged In: YES user_id=21627 I can't see the bug. Sure "AE" *has* a character (actually two of them), but it *is* not a character - a character is a string of length 1. You need such a thing on the left-hand side of 'in '. How would you formulate the error message? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 From noreply@sourceforge.net Thu Nov 29 18:30:57 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 10:30:57 -0800 Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c Message-ID: Bugs item #486434, was opened at 2001-11-28 03:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 3 Submitted By: M.-A. Lemburg (lemburg) Assigned to: M.-A. Lemburg (lemburg) Summary: Compiler complaints in posixmodule.c Initial Comment: The linker on Linux reports some warnings for posixmodule.c: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436: the use of `tempnam' is dangerous, better use `mkstemp' Perhaps we ought to follow the advice ?! ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 10:30 Message: Logged In: YES user_id=21627 We already expose os.tmpfile, I don't think we need mkstemp. I don't think we should remove tmpnam; applications that use it will get the warning (for the first time in 2.2); we should leave it to the applications to migrate away from it. I found the recommendation not to use mkstemp on a Debian 'testing' system; dunno whether it was added by the Debian maintainers, or whether it is part of more recent manpage package. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:17 Message: Logged In: YES user_id=38388 Hmm, the man page on SuSE Linux does not say anything about using tmpfile() instead of mkstemp(). BTW, the warnings are already in place. I wonder whether it wouldn't be better to remove the Python APIs for these functions altogether and instead provide interfaces for the mkstemp() and/or tempfile(). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 05:56 Message: Logged In: YES user_id=21627 No. The usage of tmpnam is not dangerous; we are just exposing it to the Python application. It may be reasonable to produce a warning if tmpnam is called. We cannot replace tempnam with mkstemp for the same reason Posix couldn't: one produces a string, the other one a file handle. What we could do is to expose mkstemp(3) where available. I don't see the value of that, though: it could be done only on systems where mkstemp is available, and we already expose tmpfile(3). In fact, the Linux man page for mkstemp(3) says # Don't use this function, use tmpfile(3) instead. It's # bet­ter defined and more portable. If you still think there is a need for action, please propose a patch. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 From noreply@sourceforge.net Thu Nov 29 18:52:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 10:52:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-487165 ] difficult to find string interpolation Message-ID: Bugs item #487165, was opened at 2001-11-29 10:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487165&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Gábor BORGULYA (borgulya) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: difficult to find string interpolation Initial Comment: I tried to look up the possibilities how to formulate expressions like 'Hello %s\n' % name After a long search I found out that this is "string interpolation". But searching for the word 'interpolation' resulted only in pages about interpolation errors. Finally I found the page: Python library reference, 2.1.5.2 String Formatting Operations. This is what I wanted to see. This page does not contain the word "interpolation". I think this part of the documentation should be made easier to find. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487165&group_id=5470 From noreply@sourceforge.net Thu Nov 29 19:05:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 11:05:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-487147 ] Differ Object doc typo Message-ID: Bugs item #487147, was opened at 2001-11-29 10:06 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487147&group_id=5470 Category: Documentation Group: Python 2.2 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Differ Object doc typo Initial Comment: In lib docs, section 4.4.3 end: "The delta generated also consists of newline- terminated strings, ready to be printed as-is via the writeline() method of a file-like object. " When writelines() method was intended. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-29 11:05 Message: Logged In: YES user_id=3066 Fixed in Doc/lib/libdifflib.tex revision 1.11. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487147&group_id=5470 From noreply@sourceforge.net Thu Nov 29 19:30:50 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 11:30:50 -0800 Subject: [Python-bugs-list] [ python-Bugs-487193 ] ftplib violates rfc1123 section 4.1.2.6 Message-ID: Bugs item #487193, was opened at 2001-11-29 11:29 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487193&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Suchandra Thapa (ssthapa) Assigned to: Nobody/Anonymous (nobody) Summary: ftplib violates rfc1123 section 4.1.2.6 Initial Comment: RFC 1123 specifies that ftp clients can not depend on 227 responses to pasv commands having the format given in the rfc 959. The ftplib assumes that the pasv parameters are returned within parentheses. In particular ftplib breaks when downloading binaries from an anonftpd servre. I've included a patch that uses the re module to obtain the pasv parameters instead. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487193&group_id=5470 From noreply@sourceforge.net Thu Nov 29 22:43:43 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 14:43:43 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Thu Nov 29 23:06:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 15:06:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Closed Resolution: Works For Me Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: M.-A. Lemburg (lemburg) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-29 15:06 Message: Logged In: YES user_id=31435 Thanks for the info, MAL! If I'm reading it right, on Windows you use an installation tree (rather than the Windows Python build tree -- it doesn't sound like you compile Python itself on Windows). ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:24 Message: Logged In: YES user_id=38388 BTW, you'll find a whole bunch of tools to "find" the Python installation in distutils. Closing the report since I don't see any point leaving it open ;-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-26 05:27 Message: Logged In: YES user_id=38388 Here's a summary of what I do to support the mx Extensions for multiple Python versions: Note: I use distutils on all platforms and distutils has builtin ways to find where the specific Python version I am targetting lives. On Unix: Since the only binary format distutils currently support is RPM, this is only relevant to the bdist_rpm command. Unfortunately, RPM does not inherit the environment, so you have to hard-code the path to the Python version into the RPM. It goes something like this: $(PYTHON) setup.py bdist_rpm \ --python $(PYTHON) \ --release $(PACKAGERELEASE) \ $(_RPM_REQUIRES) On Windows: Here, you have to run the setup.py file using the Python version you wish to build for, e.g. python2.0 setup.py bdist_win. I choose the Python version by hacking the PATH to point to the right version (this is needed to have sub-processes use the same Python version), e.g. wininst20.bat: @echo off call vcvars32 echo -------------------------------------------------------------- echo Building for Python 2.0... set PATH="d:\Python20;%PATH%" python setup.py clean --all python setup.py bdist_wininst -c -o That's about it. Hope it helps. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-23 18:16 Message: Logged In: YES user_id=31435 It clarifies the problem but mainly in the sense of clarifying that it *is* a problem: we don't require, or even recommend, any particular way of building extensions. IOW, "but I can't be certain" is correct, but not for a reason I can do anything about except to confirm that it's the truth. Marc-Andre Lemburg is our most prolific extension writer, so I'm reassigning this to him in the hope he'll explain what *he* does. You could do a lot worse than catering to whatever that is. ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 15:34 Message: Logged In: YES user_id=52572 The problem goes something like this: Assume I have a nice, generalized build system (I do, but not as nice as it will be soon ;-)), to which I want to add Python support. I want it to be possible to compile/test code against multiple Python versions, though that's not too relevant to the problem. I'd like finding the neccessary components (headers, libs, executables) to be relatively automatic, based on, e.g. the user's specification of a python base directory and a version number. On Unices it should be possible to do without the base directory unless python was configured --with-prefix=.... I have to assume that someone developing Python extension modules may either be using the source tree, or an installation specific to to the platform he's developing on. Someone who's just building an extension module is probably using a Python installation, but I can't be certain. So, does that clear things up a bit? -Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-15 15:15 Message: Logged In: YES user_id=31435 The source distro is just a snapshot of Python's CVS development tree, so it's a build tree, not an installation tree. "The std" Windows installation tree is the one created by the PythonLabs Windows installer. Similar Windows trees are created by the ActiveState and PythonWare Windows installers. "The std" Unix installation tree is whatever the heck "make install" does on Unix (and it's not the *build* tree either). Likewise for Macs, which rearrage the build tree in yet other ways for installation. The relationship between the build tree and the Windows installation tree is established by the 3000-line python20.wse script in PCbuild, which is input to the Wise installer-builder; I can't summarize it usefully here, and Windows installation also involves fiddling with registry entries. Afraid I'm still not clear on why you're asking, although I'm quite sure I'm not really helping . ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Thu Nov 29 23:46:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 15:46:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-487297 ] Copy from stdout after crash Message-ID: Bugs item #487297, was opened at 2001-11-29 15:46 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487297&group_id=5470 Category: Macintosh Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Jack Jansen (jackjansen) Summary: Copy from stdout after crash Initial Comment: It would be really nice if one could copy text from the stdout window (e.g. PythonInterpreter.out) after a crash. Apparently this now works in some cases, but still fails after a crash in a delay-console-window applet. I am submitting this to SourceForge as per Jack Jansens' request. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487297&group_id=5470 From noreply@sourceforge.net Thu Nov 29 23:51:34 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 15:51:34 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:51 Message: Logged In: YES user_id=21627 Thansk for the files. I now trying to figure out why Python refuses to use the getaddrinfo implementation in your C library. It may be that it is really broken, in which case we must make getaddrinfo.c work correctly. Perhaps it ought to work,and configure just fails to correctly see that. So please compile the attached file aicheck.c on your system, and report output and status code (looking at $?); if it fails to compile, report the compile errors. ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-28 12:21 Message: Logged In: YES user_id=85984 Attached are three files: - pyconfig.h from a fresh gcc 2.7.2.1 build - config.log from a fresh gcc 2.7.2.1 build - /usr/include/netdb.h - the getaddrinfo manual page from my system Indeed, EAI_ADDRFAMILY is defined in /usr/include/netdb.h, but EAI_MAX is not. Let me know if I can provide any more information. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 01:30 Message: Logged In: YES user_id=21627 Ok, then we need to investigate the configure results. Can you please attach pyconfig.h and config.log to this report? Also, can you please confirm that you build Python from scratch with 2.7.2.1 (i.e untarring a fresh source tree, to avoid using a config.cache left over from gcc 2.95.2)? Furthermore, can you please report details of the getaddrinfo implementation on your system? Does it provide one? In the C library? If not, can you confirm that EAI_ADDRFAMILY is defined in your system headers, but EAI_MAX is not? If yes, can you speculate as to why configure thinks your system does not have getaddrinfo? ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-27 15:21 Message: Logged In: YES user_id=85984 I probably should have included the output from gcc 2.7.2.1 instead which I know is properly installed. The results are similar, although there is no mention of `wint_t': % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from Include/pyport.h:93, from Include/Python.h:60, from ./Modules/socketmodule.c:78: /usr/include/math.h:177: warning: function declaration isn't a prototype /usr/include/math.h:257: warning: function declaration isn't a prototype In file included from Include/unicodeobject.h:118, from Include/Python.h:69, from ./Modules/socketmodule.c:78: /usr/include/wchar.h:15: warning: function declaration isn't a prototype In file included from /usr/include/signal.h:46, from ./Modules/socketmodule.c:140: /usr/include/sys/signal.h:121: warning: function declaration isn't a prototype /usr/include/sys/signal.h:164: warning: function declaration isn't a prototype Modules/getaddrinfo.c: In function `gai_strerror': In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Thu Nov 29 23:53:03 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 15:53:03 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 00:10:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 16:10:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 00:27:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 16:27:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-487308 ] Tkinter class links broken/missing Message-ID: Bugs item #487308, was opened at 2001-11-29 16:27 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487308&group_id=5470 Category: Documentation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Greg Chapman (glchapman) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Tkinter class links broken/missing Initial Comment: This page of the development documentation: http://www.python.org/doc/2.2/lib/node500.html has links which should lead to the interfaces of the widgets listed. However, the links result in 404 errors. (The individual widget pages also appear to be missing from the Windows 2.2b2 distribution. At any rate, they are missing from the HTML help file I compiled using the .html files from the distribution). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487308&group_id=5470 From noreply@sourceforge.net Fri Nov 30 00:35:19 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 16:35:19 -0800 Subject: [Python-bugs-list] [ python-Bugs-487310 ] smtplib mishandles SMTP disconnects Message-ID: Bugs item #487310, was opened at 2001-11-29 16:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487310&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: smtplib mishandles SMTP disconnects Initial Comment: smtplib handles SMTP disconnects (e.g. timeouts) inconsistently. The smtplib.SMTP.send() method does not reset self.file on a socket error, unlike getreply(), and thus a close() is necessary for in one case but not in the other. Recommendation: have smtplib.SMTP.send() call close() on socket.error just as smtplib.SMTP.getreply() does on EOF on self.file. In the following test case, I have set the SMTP timeout on my local machine to 10 seconds: Python 2.1.1 (#1, Nov 7 2001, 16:18:10) [GCC 2.95.3 20010315 (release)] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import smtplib >>> s = smtplib.SMTP('localhost', 25) >>> import time >>> time.sleep(10) >>> s.sendmail('fmajid@kefta.com', 'fmajid@kefta.com', '...') Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/smtplib.py", line 469, in sendmail (code,resp) = self.helo() File "/usr/local/lib/python2.1/smtplib.py", line 305, in helo self.putcmd("helo", socket.getfqdn()) File "/usr/local/lib/python2.1/smtplib.py", line 249, in putcmd self.send(str) File "/usr/local/lib/python2.1/smtplib.py", line 239, in send raise SMTPServerDisconnected('Server not connected') smtplib.SMTPServerDisconnected: Server not connected >>> s.connect('localhost', 25) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/smtplib.py", line 226, in connect (code,msg)=self.getreply() File "/usr/local/lib/python2.1/smtplib.py", line 271, in getreply raise SMTPServerDisconnected("Connection unexpectedly closed") smtplib.SMTPServerDisconnected: Connection unexpectedly closed >>> s.connect('localhost', 25) (220, 'bayazid.kefta.com ESMTP Postfix') ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487310&group_id=5470 From noreply@sourceforge.net Fri Nov 30 01:12:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 17:12:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Nobody/Anonymous (nobody) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 From noreply@sourceforge.net Fri Nov 30 01:33:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 17:33:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-487321 ] Installation fails on module-xmllib.html Message-ID: Bugs item #487321, was opened at 2001-11-29 17:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Nobody/Anonymous (nobody) Summary: Installation fails on module-xmllib.html Initial Comment: On NT 4.0 SP6 installation of Python 2.2b2 hangs with the following message: "Copying Python Documentation (HTML); F:\Python22\Doc\lib\module-xmllib.html Upon further investigation it appears the installation program is getting hung up while it creates an infinite length .tmp file (multiple Gb) that consumes all available disk space on the installation drive. The installation file Python-2.2b2.exe has a file length of 7,008,533 bytes. I've now tried to install both versions 2.1 and 2.2b2 - both installations have failed for different reasons. Please get the Windows installations fixed! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 From noreply@sourceforge.net Fri Nov 30 02:09:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 18:09:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-487331 ] time mod's timezone doesn't honor TZ var Message-ID: Bugs item #487331, was opened at 2001-11-29 18:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Nobody/Anonymous (nobody) Summary: time mod's timezone doesn't honor TZ var Initial Comment: Python's time module seems to get the timezone name when it's first imported, but it doesn't change it if the TZ environment variable is later set. On the other hand, the time of day DOES change accordingly. Thus, setting TZ after the time module is imported results in a completely broken rfc2822 date - time in the new TZ, but with a bogus UTC offset and timezone (neither the old nor new). Here is the problem illustrated. You will see that after setting TZ, altzone, timezone, and tzname don't change, while asctime does. $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import time,os >>> print time.asctime() Thu Nov 29 18:47:39 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print time.asctime() Fri Nov 30 14:47:52 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') Here is an example of why this is a problem in practice. One of my applications sets the TZ environment variable based on a "TIMEZONE" setting in the user's configuration file. It later uses the `email' module to create a "Date:" header for outgoing mail messages. Because of this bug, the resulting "Date:" is bogus. e.g, $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import email,os >>> print email.Utils.formatdate(localtime=1) Thu, 29 Nov 2001 18:52:34 -0700 >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print email.Utils.formatdate(localtime=1) Fri, 30 Nov 2001 14:52:41 -0600 The `-0600' should be `+1300', and `-0600' isn't correct even for the original timezone! You might say: "Work around this by setting TZ before the time module is first imported." -- No, that is too fragile, and too difficult to do with complex applications. Many other modules in turn import time, etc. I suggest instead that the TZ environment variable be consulted each time the timezone is accessed. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 From noreply@sourceforge.net Fri Nov 30 02:18:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 18:18:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-29 18:18 Message: Logged In: YES user_id=85984 aicheck.c compiled without problem on my system. % gcc -o aicheck aicheck.c % Running the resulting binary produces no output. I'm not exactly sure what kind of output you are looking for (taking a guess): % gdb /home/jason/temp/aicheck GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.16 (i386-unknown-bsdi4.0), Copyright 1996 Free Software Foundation, Inc...(no debugging symbols found)... (gdb) run Starting program: /home/jason/temp/aicheck (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... Program exited with code 01. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:51 Message: Logged In: YES user_id=21627 Thansk for the files. I now trying to figure out why Python refuses to use the getaddrinfo implementation in your C library. It may be that it is really broken, in which case we must make getaddrinfo.c work correctly. Perhaps it ought to work,and configure just fails to correctly see that. So please compile the attached file aicheck.c on your system, and report output and status code (looking at $?); if it fails to compile, report the compile errors. ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-28 12:21 Message: Logged In: YES user_id=85984 Attached are three files: - pyconfig.h from a fresh gcc 2.7.2.1 build - config.log from a fresh gcc 2.7.2.1 build - /usr/include/netdb.h - the getaddrinfo manual page from my system Indeed, EAI_ADDRFAMILY is defined in /usr/include/netdb.h, but EAI_MAX is not. Let me know if I can provide any more information. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 01:30 Message: Logged In: YES user_id=21627 Ok, then we need to investigate the configure results. Can you please attach pyconfig.h and config.log to this report? Also, can you please confirm that you build Python from scratch with 2.7.2.1 (i.e untarring a fresh source tree, to avoid using a config.cache left over from gcc 2.95.2)? Furthermore, can you please report details of the getaddrinfo implementation on your system? Does it provide one? In the C library? If not, can you confirm that EAI_ADDRFAMILY is defined in your system headers, but EAI_MAX is not? If yes, can you speculate as to why configure thinks your system does not have getaddrinfo? ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-27 15:21 Message: Logged In: YES user_id=85984 I probably should have included the output from gcc 2.7.2.1 instead which I know is properly installed. The results are similar, although there is no mention of `wint_t': % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from Include/pyport.h:93, from Include/Python.h:60, from ./Modules/socketmodule.c:78: /usr/include/math.h:177: warning: function declaration isn't a prototype /usr/include/math.h:257: warning: function declaration isn't a prototype In file included from Include/unicodeobject.h:118, from Include/Python.h:69, from ./Modules/socketmodule.c:78: /usr/include/wchar.h:15: warning: function declaration isn't a prototype In file included from /usr/include/signal.h:46, from ./Modules/socketmodule.c:140: /usr/include/sys/signal.h:121: warning: function declaration isn't a prototype /usr/include/sys/signal.h:164: warning: function declaration isn't a prototype Modules/getaddrinfo.c: In function `gai_strerror': In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Fri Nov 30 02:25:49 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 18:25:49 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) >Assigned to: Tim Peters (tim_one) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:25 Message: Logged In: YES user_id=31435 Tens of thousands of Windows users have run this installer without problems, so-- and especially given your other bug report about 2.2b2 --the presumption has to be that something is uniquely screwed up on the specific box you tried it on. Can you reproduce on another (physically distinct) machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 From noreply@sourceforge.net Fri Nov 30 02:27:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 18:27:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-487321 ] Installation fails on module-xmllib.html Message-ID: Bugs item #487321, was opened at 2001-11-29 17:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) >Assigned to: Tim Peters (tim_one) Summary: Installation fails on module-xmllib.html Initial Comment: On NT 4.0 SP6 installation of Python 2.2b2 hangs with the following message: "Copying Python Documentation (HTML); F:\Python22\Doc\lib\module-xmllib.html Upon further investigation it appears the installation program is getting hung up while it creates an infinite length .tmp file (multiple Gb) that consumes all available disk space on the installation drive. The installation file Python-2.2b2.exe has a file length of 7,008,533 bytes. I've now tried to install both versions 2.1 and 2.2b2 - both installations have failed for different reasons. Please get the Windows installations fixed! ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:27 Message: Logged In: YES user_id=31435 See comment on your bug 487317: you're unique here too. Can you reproduce on a physically distinct machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 From noreply@sourceforge.net Fri Nov 30 04:02:36 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 20:02:36 -0800 Subject: [Python-bugs-list] [ python-Bugs-486826 ] bogus error message on bad parameters Message-ID: Bugs item #486826, was opened at 2001-11-28 20:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bogus error message on bad parameters Initial Comment: I was writing a CGI and ran across the following error message: TypeError: 'in ' requires character as left operand, which drove me batty as I keep thinking it was the left operand when in reality it was the right operand.... I've simplified the problem to Python 2.1.1 (#1, Oct 30 2001, 09:41:48) [GCC 2.95.2 19991024 (release)] on darwin1 Type "copyright", "credits" or "license" for more information. >>> mystrings = "AE,Developer" >>> "AE" in mystrings Traceback (most recent call last): File "", line 1, in ? TypeError: 'in ' requires character as left operand >>> maybe it's just me but I keep thinking that "AE" is the left operand and it has a character.... ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-29 19:59 Message: Logged In: NO Ok, I played with the code some more.... and learned a thing or two... I now realize that the "in" command can be used on non lists such as "a in 'abcd'" whereas I was trying to do "'AE' in ['AE','Developer']" as part of the CGI. Python v1.5.2 didn't help that much as the error reported was "TypeError: string member test needs char left operand". Had I had my thinking cap on, I would have seen the difference in the Python 2.1.1 error message 'in < string>' which would have indicated that the right hand side was a string vs the list I was expecting... Now knowing this, I think it's safe that we can close this bug. Thanks for teaching me something new today! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 10:23 Message: Logged In: YES user_id=21627 I can't see the bug. Sure "AE" *has* a character (actually two of them), but it *is* not a character - a character is a string of length 1. You need such a thing on the left-hand side of 'in '. How would you formulate the error message? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 From noreply@sourceforge.net Fri Nov 30 05:10:32 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 21:10:32 -0800 Subject: [Python-bugs-list] [ python-Bugs-487390 ] Modifying type.__module__ behavior Message-ID: Bugs item #487390, was opened at 2001-11-29 21:10 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487390&group_id=5470 Category: Python Library Group: Feature Request Status: Open Resolution: None Priority: 5 Submitted By: Burton Radons (loth) Assigned to: Nobody/Anonymous (nobody) Summary: Modifying type.__module__ behavior Initial Comment: A __module__ attribute request on a C type will return __builtin__ if the typeobject.c code cannot find a period in the type name. This is always incorrect for extension libraries, and can be a _very_ confusing error when trying to pickle your types (as the error makes it seem as if you're supposed to register your type in __builtin__). I propose that: A) All builtin Python types have a "__builtin__." prefixed to their type declaration's name. B) If the __module__ code cannot find a period, it raises AttributeError with a somewhat descriptive message. This would be in the place where it returns __builtin__. C) The tp_name comment in Include/object.h describe the special semantics. "/* 'module.typename' */", for example. This is a lot of files to change, I know, but it's only one or two line changes each file. The only code this should affect is code which is written incorrectly; it may not understand the name semantics, or it may actually be registering itself in __builtin__ (as I did before I figured out what it was doing). Otherwise we're golden. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487390&group_id=5470 From noreply@sourceforge.net Fri Nov 30 06:07:53 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Thu, 29 Nov 2001 22:07:53 -0800 Subject: [Python-bugs-list] [ python-Bugs-458447 ] New httplib lacks documentation Message-ID: Bugs item #458447, was opened at 2001-09-04 10:01 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458447&group_id=5470 Category: Documentation Group: None >Status: Closed >Resolution: Fixed Priority: 6 Submitted By: Rich Salz (rsalz) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: New httplib lacks documentation Initial Comment: urllib should allow the user to set the content-type. This would allow applications to do more than just "forms posting" -- e.g., use it to make SOAP requests. The attached patch allows that. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-29 22:07 Message: Logged In: YES user_id=3066 Checked in as Doc/lib/libhttplib.tex revision 1.28. Updates contributed by Kalle Svensson. Thanks, Kalle! ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-11-26 13:51 Message: Logged In: YES user_id=3066 I have contributed docs for the new version waiting for review in my inbox; these should be checked in this week. Bumping priority so this stays visible to me. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-05 08:37 Message: Logged In: YES user_id=6380 Fred: it turns out the httplib docs still document the old version of the module. The new version is considerably more powerful, and is essentially undocumented. Maybe you can shame Greg Stein into providing some docs, or maybe you can convert the copious docstrings to LaTeX. ---------------------------------------------------------------------- Comment By: Skip Montanaro (montanaro) Date: 2001-09-04 20:45 Message: Logged In: YES user_id=44345 I'm not sure why you'd want to program SOAP (or XMLRPC, for that matter) directly. You would (normally) only use those protocols through special-purpose modules like SOAP.py or xmlrpclib.py, both of which talk to httplib I believe. I don't see that allowing the Content-Type: header to be overridden at the urllib level is all that necessary. If you're going to want to mess with Content-Type: you're probably going to want to mess with other headers as well. ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-09-04 19:02 Message: Logged In: YES user_id=36737 I created this bug because urllib didn't let you set the content-type header. Guido pointed out that I should be using httplib. I didn't use httplib because I didn't know until I read the source, that httplib had an https object that used SSL. So I guess "document the https object" is what this has mutated into. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2001-09-04 14:03 Message: Logged In: YES user_id=3066 It's not at all clear what sort of documentation update is needed. What did you have in mind? ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-04 12:20 Message: Logged In: YES user_id=6380 Please don't Delete requests. Please don't Close them unless you are a project manager (I consider it a SF bug that the submitter can decide to close a bug report if they happen to have admin perms on another project). I've reopened this and recategorized as Doc and assigned to Fred so he can see if the httplib docs need to be updated. ---------------------------------------------------------------------- Comment By: Rich Salz (rsalz) Date: 2001-09-04 12:16 Message: Logged In: YES user_id=36737 Okay, I'll accept that. Had HTTP documented that it included HTTPS I wouldn't have made the initial mistake. :) Let me know if you want the diff. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-04 11:20 Message: Logged In: YES user_id=6380 Isn't SOAP an HTTP application? Then you shouldn't be using urllib, but httplib. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2001-09-04 11:20 Message: Logged In: YES user_id=6380 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=458447&group_id=5470 From noreply@sourceforge.net Fri Nov 30 09:19:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 01:19:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:19 Message: Logged In: YES user_id=21627 Any change is user-visible: for any change, I can construct a program that behaves differently with the change. Otherwise, the change would be useless (except that it may add commentary or some such). In fact, the change *is* documented, namely in the CVS log. Extracting all those fragments into a single document would be time-consuming and pointless: nobody can read through hundreds of changes. Your program would have blown up even if there was such a document. If a change has severe effects on many existing programs, it should be implemented differently. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 09:35:48 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 01:35:48 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:35 Message: Logged In: YES user_id=21627 Arrghh. I was trying to find out which of the tests fail, but attached the wrong version of the file: I meant to put printfs into each failure. Please try aicheck2.c instead. Anyway, it seems that your system is indeed one of those where the test should find out problems with the C library (unless the test that fails is bogus, which we can investigate once we know what it is that fails). I'll try to come up with a patch that allows usage of the getaddrinfo emulation even on systems where getaddrinfo is in the C library. ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-29 18:18 Message: Logged In: YES user_id=85984 aicheck.c compiled without problem on my system. % gcc -o aicheck aicheck.c % Running the resulting binary produces no output. I'm not exactly sure what kind of output you are looking for (taking a guess): % gdb /home/jason/temp/aicheck GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.16 (i386-unknown-bsdi4.0), Copyright 1996 Free Software Foundation, Inc...(no debugging symbols found)... (gdb) run Starting program: /home/jason/temp/aicheck (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... Program exited with code 01. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:51 Message: Logged In: YES user_id=21627 Thansk for the files. I now trying to figure out why Python refuses to use the getaddrinfo implementation in your C library. It may be that it is really broken, in which case we must make getaddrinfo.c work correctly. Perhaps it ought to work,and configure just fails to correctly see that. So please compile the attached file aicheck.c on your system, and report output and status code (looking at $?); if it fails to compile, report the compile errors. ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-28 12:21 Message: Logged In: YES user_id=85984 Attached are three files: - pyconfig.h from a fresh gcc 2.7.2.1 build - config.log from a fresh gcc 2.7.2.1 build - /usr/include/netdb.h - the getaddrinfo manual page from my system Indeed, EAI_ADDRFAMILY is defined in /usr/include/netdb.h, but EAI_MAX is not. Let me know if I can provide any more information. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 01:30 Message: Logged In: YES user_id=21627 Ok, then we need to investigate the configure results. Can you please attach pyconfig.h and config.log to this report? Also, can you please confirm that you build Python from scratch with 2.7.2.1 (i.e untarring a fresh source tree, to avoid using a config.cache left over from gcc 2.95.2)? Furthermore, can you please report details of the getaddrinfo implementation on your system? Does it provide one? In the C library? If not, can you confirm that EAI_ADDRFAMILY is defined in your system headers, but EAI_MAX is not? If yes, can you speculate as to why configure thinks your system does not have getaddrinfo? ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-27 15:21 Message: Logged In: YES user_id=85984 I probably should have included the output from gcc 2.7.2.1 instead which I know is properly installed. The results are similar, although there is no mention of `wint_t': % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from Include/pyport.h:93, from Include/Python.h:60, from ./Modules/socketmodule.c:78: /usr/include/math.h:177: warning: function declaration isn't a prototype /usr/include/math.h:257: warning: function declaration isn't a prototype In file included from Include/unicodeobject.h:118, from Include/Python.h:69, from ./Modules/socketmodule.c:78: /usr/include/wchar.h:15: warning: function declaration isn't a prototype In file included from /usr/include/signal.h:46, from ./Modules/socketmodule.c:140: /usr/include/sys/signal.h:121: warning: function declaration isn't a prototype /usr/include/sys/signal.h:164: warning: function declaration isn't a prototype Modules/getaddrinfo.c: In function `gai_strerror': In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Fri Nov 30 09:36:20 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 01:36:20 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:36 Message: Logged In: YES user_id=38388 I disagree: any change which could potentially blow up existing programs should at least be mentioned somewhere in the docs, be it the NEWS file, the release page on python.org or (thanks to the great work of Andrew Kuchling) the "What's new in Python x.x" docs. This is simply needed due to the very dynamic way Python works: there's no way to test all paths through a program if you want to port it from one Python version to the next, so we'll have to be very careful about these changes even if they are clearly bug fixes. In the mentioned case, I'd say that the programmer was at fault, though: it's so much easier to test a path for being a directory than to rely on some obscure method return value and also much safer ! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:19 Message: Logged In: YES user_id=21627 Any change is user-visible: for any change, I can construct a program that behaves differently with the change. Otherwise, the change would be useless (except that it may add commentary or some such). In fact, the change *is* documented, namely in the CVS log. Extracting all those fragments into a single document would be time-consuming and pointless: nobody can read through hundreds of changes. Your program would have blown up even if there was such a document. If a change has severe effects on many existing programs, it should be implemented differently. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 09:37:35 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 01:37:35 -0800 Subject: [Python-bugs-list] [ python-Bugs-481972 ] Explanation of installation Message-ID: Bugs item #481972, was opened at 2001-11-14 21:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 Category: Installation Group: Feature Request Status: Closed Resolution: Works For Me Priority: 5 Submitted By: David Abrahams (david_abrahams) Assigned to: M.-A. Lemburg (lemburg) Summary: Explanation of installation Initial Comment: Can someone describe for me the relationship between the directory structure that comes in the source distribution and what I get on my disk when installed under Windows? Some things are not in the same place, and I can't make rhyme nor reason of it. Thanks, Dave david.abrahams@rcn.com ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:37 Message: Logged In: YES user_id=38388 Right. In fact, I've never compiled Python on Windows in all the years. You've done such a wonderful job that this simply isn't necessary :-) ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 15:06 Message: Logged In: YES user_id=31435 Thanks for the info, MAL! If I'm reading it right, on Windows you use an installation tree (rather than the Windows Python build tree -- it doesn't sound like you compile Python itself on Windows). ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:24 Message: Logged In: YES user_id=38388 BTW, you'll find a whole bunch of tools to "find" the Python installation in distutils. Closing the report since I don't see any point leaving it open ;-) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-26 05:27 Message: Logged In: YES user_id=38388 Here's a summary of what I do to support the mx Extensions for multiple Python versions: Note: I use distutils on all platforms and distutils has builtin ways to find where the specific Python version I am targetting lives. On Unix: Since the only binary format distutils currently support is RPM, this is only relevant to the bdist_rpm command. Unfortunately, RPM does not inherit the environment, so you have to hard-code the path to the Python version into the RPM. It goes something like this: $(PYTHON) setup.py bdist_rpm \ --python $(PYTHON) \ --release $(PACKAGERELEASE) \ $(_RPM_REQUIRES) On Windows: Here, you have to run the setup.py file using the Python version you wish to build for, e.g. python2.0 setup.py bdist_win. I choose the Python version by hacking the PATH to point to the right version (this is needed to have sub-processes use the same Python version), e.g. wininst20.bat: @echo off call vcvars32 echo -------------------------------------------------------------- echo Building for Python 2.0... set PATH="d:\Python20;%PATH%" python setup.py clean --all python setup.py bdist_wininst -c -o That's about it. Hope it helps. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-23 18:16 Message: Logged In: YES user_id=31435 It clarifies the problem but mainly in the sense of clarifying that it *is* a problem: we don't require, or even recommend, any particular way of building extensions. IOW, "but I can't be certain" is correct, but not for a reason I can do anything about except to confirm that it's the truth. Marc-Andre Lemburg is our most prolific extension writer, so I'm reassigning this to him in the hope he'll explain what *he* does. You could do a lot worse than catering to whatever that is. ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 15:34 Message: Logged In: YES user_id=52572 The problem goes something like this: Assume I have a nice, generalized build system (I do, but not as nice as it will be soon ;-)), to which I want to add Python support. I want it to be possible to compile/test code against multiple Python versions, though that's not too relevant to the problem. I'd like finding the neccessary components (headers, libs, executables) to be relatively automatic, based on, e.g. the user's specification of a python base directory and a version number. On Unices it should be possible to do without the base directory unless python was configured --with-prefix=.... I have to assume that someone developing Python extension modules may either be using the source tree, or an installation specific to to the platform he's developing on. Someone who's just building an extension module is probably using a Python installation, but I can't be certain. So, does that clear things up a bit? -Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-15 15:15 Message: Logged In: YES user_id=31435 The source distro is just a snapshot of Python's CVS development tree, so it's a build tree, not an installation tree. "The std" Windows installation tree is the one created by the PythonLabs Windows installer. Similar Windows trees are created by the ActiveState and PythonWare Windows installers. "The std" Unix installation tree is whatever the heck "make install" does on Unix (and it's not the *build* tree either). Likewise for Macs, which rearrage the build tree in yet other ways for installation. The relationship between the build tree and the Windows installation tree is established by the 3000-line python20.wse script in PCbuild, which is input to the Wise installer-builder; I can't summarize it usefully here, and Windows installation also involves fiddling with registry entries. Afraid I'm still not clear on why you're asking, although I'm quite sure I'm not really helping . ---------------------------------------------------------------------- Comment By: David Abrahams (david_abrahams) Date: 2001-11-15 04:12 Message: Logged In: YES user_id=52572 Hi Tim, Usually I install using the windows installer, and eventually need to get the source so I download it elsewhere; then I usually need the "_d" version so I build that from the source. The two "installations" seem different to me, and I'm not certain of all the ways in which they differ. Just for example, when I build on Windows, the executable(s) end up in the PCBuild subdirectory. Is there a standard for how a Windows installation should look? Why do I care? I've been making a build/test install system, which needs to link with/invoke python. Thanks, Dave ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-14 21:39 Message: Logged In: YES user_id=31435 David, I'm not clear on what "installed" means to you. Are you building from source yourself, or using the PythonLabs Windows installer? If the latter, the Windows distribution is a precompiled distribution of a subset of the source tree, plus a number of precompiled "external" packages (like bsddb, zlib, expat, and Tcl/Tk). The Python source distro is pruned on Windows to reflect that a vast majority of our Windows users have no compiler, and no interest in anything outside of Lib/ and a few of the Tools/ directories. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=481972&group_id=5470 From noreply@sourceforge.net Fri Nov 30 09:38:08 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 01:38:08 -0800 Subject: [Python-bugs-list] [ python-Bugs-486826 ] bogus error message on bad parameters Message-ID: Bugs item #486826, was opened at 2001-11-28 20:56 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 Category: Python Interpreter Core Group: None >Status: Closed >Resolution: Wont Fix Priority: 5 Submitted By: Nobody/Anonymous (nobody) Assigned to: Nobody/Anonymous (nobody) Summary: bogus error message on bad parameters Initial Comment: I was writing a CGI and ran across the following error message: TypeError: 'in ' requires character as left operand, which drove me batty as I keep thinking it was the left operand when in reality it was the right operand.... I've simplified the problem to Python 2.1.1 (#1, Oct 30 2001, 09:41:48) [GCC 2.95.2 19991024 (release)] on darwin1 Type "copyright", "credits" or "license" for more information. >>> mystrings = "AE,Developer" >>> "AE" in mystrings Traceback (most recent call last): File "", line 1, in ? TypeError: 'in ' requires character as left operand >>> maybe it's just me but I keep thinking that "AE" is the left operand and it has a character.... ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2001-11-29 19:59 Message: Logged In: NO Ok, I played with the code some more.... and learned a thing or two... I now realize that the "in" command can be used on non lists such as "a in 'abcd'" whereas I was trying to do "'AE' in ['AE','Developer']" as part of the CGI. Python v1.5.2 didn't help that much as the error reported was "TypeError: string member test needs char left operand". Had I had my thinking cap on, I would have seen the difference in the Python 2.1.1 error message 'in < string>' which would have indicated that the right hand side was a string vs the list I was expecting... Now knowing this, I think it's safe that we can close this bug. Thanks for teaching me something new today! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 10:23 Message: Logged In: YES user_id=21627 I can't see the bug. Sure "AE" *has* a character (actually two of them), but it *is* not a character - a character is a string of length 1. You need such a thing on the left-hand side of 'in '. How would you formulate the error message? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486826&group_id=5470 From noreply@sourceforge.net Fri Nov 30 09:40:16 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 01:40:16 -0800 Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c Message-ID: Bugs item #486434, was opened at 2001-11-28 03:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 3 Submitted By: M.-A. Lemburg (lemburg) Assigned to: M.-A. Lemburg (lemburg) Summary: Compiler complaints in posixmodule.c Initial Comment: The linker on Linux reports some warnings for posixmodule.c: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436: the use of `tempnam' is dangerous, better use `mkstemp' Perhaps we ought to follow the advice ?! ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:40 Message: Logged In: YES user_id=38388 Ok. How about this: we produce warnings for the two APIs in question in 2.2 and drop their support in 2.3 ?! I hate seeing the linker warn me about Python using dangerous system APIs -- this simply doesn't look right. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 10:30 Message: Logged In: YES user_id=21627 We already expose os.tmpfile, I don't think we need mkstemp. I don't think we should remove tmpnam; applications that use it will get the warning (for the first time in 2.2); we should leave it to the applications to migrate away from it. I found the recommendation not to use mkstemp on a Debian 'testing' system; dunno whether it was added by the Debian maintainers, or whether it is part of more recent manpage package. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:17 Message: Logged In: YES user_id=38388 Hmm, the man page on SuSE Linux does not say anything about using tmpfile() instead of mkstemp(). BTW, the warnings are already in place. I wonder whether it wouldn't be better to remove the Python APIs for these functions altogether and instead provide interfaces for the mkstemp() and/or tempfile(). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 05:56 Message: Logged In: YES user_id=21627 No. The usage of tmpnam is not dangerous; we are just exposing it to the Python application. It may be reasonable to produce a warning if tmpnam is called. We cannot replace tempnam with mkstemp for the same reason Posix couldn't: one produces a string, the other one a file handle. What we could do is to expose mkstemp(3) where available. I don't see the value of that, though: it could be done only on systems where mkstemp is available, and we already expose tmpfile(3). In fact, the Linux man page for mkstemp(3) says # Don't use this function, use tmpfile(3) instead. It's # bet­ter defined and more portable. If you still think there is a need for action, please propose a patch. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 From noreply@sourceforge.net Fri Nov 30 12:21:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 04:21:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 04:21 Message: Logged In: YES user_id=21627 Well, I wouldn't object to anybody documenting this particular change (even though I won't do that myself, either). I'm just pointing out that there are many more changes of this kind, and that it would be an illusion that you could ever produce a complete list. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:36 Message: Logged In: YES user_id=38388 I disagree: any change which could potentially blow up existing programs should at least be mentioned somewhere in the docs, be it the NEWS file, the release page on python.org or (thanks to the great work of Andrew Kuchling) the "What's new in Python x.x" docs. This is simply needed due to the very dynamic way Python works: there's no way to test all paths through a program if you want to port it from one Python version to the next, so we'll have to be very careful about these changes even if they are clearly bug fixes. In the mentioned case, I'd say that the programmer was at fault, though: it's so much easier to test a path for being a directory than to rely on some obscure method return value and also much safer ! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:19 Message: Logged In: YES user_id=21627 Any change is user-visible: for any change, I can construct a program that behaves differently with the change. Otherwise, the change would be useless (except that it may add commentary or some such). In fact, the change *is* documented, namely in the CVS log. Extracting all those fragments into a single document would be time-consuming and pointless: nobody can read through hundreds of changes. Your program would have blown up even if there was such a document. If a change has severe effects on many existing programs, it should be implemented differently. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 12:33:27 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 04:33:27 -0800 Subject: [Python-bugs-list] [ python-Bugs-486434 ] Compiler complaints in posixmodule.c Message-ID: Bugs item #486434, was opened at 2001-11-28 03:51 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 Category: Extension Modules Group: None Status: Open Resolution: None Priority: 3 Submitted By: M.-A. Lemburg (lemburg) Assigned to: M.-A. Lemburg (lemburg) Summary: Compiler complaints in posixmodule.c Initial Comment: The linker on Linux reports some warnings for posixmodule.c: libpython2.2.a(posixmodule.o): In function `posix_tmpnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4486: the use of `tmpnam_r' is dangerous, better use `mkstemp' libpython2.2.a(posixmodule.o): In function `posix_tempnam': /home/lemburg/projects/Python/Dev-Python/./Modules/posixmodule.c:4436: the use of `tempnam' is dangerous, better use `mkstemp' Perhaps we ought to follow the advice ?! ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 04:33 Message: Logged In: YES user_id=21627 Before they can be dropped, they must be deprecated. In this case, I see no real reason to deprecate them: They produce a warning indicating a potential problem. For some applications, there may not be a problem at all, e.g. if they write the temporary files to a directory where nobody else has write access, or if they open the temporary file with O_EXCL. There *are* ways to use tempnam safely. I don't think we should change Python just because of a stupid linker warning (which isn't stupid in general, since it made you aware of the problem - but it is unfortunate that it cannot be turned off). ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:40 Message: Logged In: YES user_id=38388 Ok. How about this: we produce warnings for the two APIs in question in 2.2 and drop their support in 2.3 ?! I hate seeing the linker warn me about Python using dangerous system APIs -- this simply doesn't look right. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 10:30 Message: Logged In: YES user_id=21627 We already expose os.tmpfile, I don't think we need mkstemp. I don't think we should remove tmpnam; applications that use it will get the warning (for the first time in 2.2); we should leave it to the applications to migrate away from it. I found the recommendation not to use mkstemp on a Debian 'testing' system; dunno whether it was added by the Debian maintainers, or whether it is part of more recent manpage package. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-28 06:17 Message: Logged In: YES user_id=38388 Hmm, the man page on SuSE Linux does not say anything about using tmpfile() instead of mkstemp(). BTW, the warnings are already in place. I wonder whether it wouldn't be better to remove the Python APIs for these functions altogether and instead provide interfaces for the mkstemp() and/or tempfile(). ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 05:56 Message: Logged In: YES user_id=21627 No. The usage of tmpnam is not dangerous; we are just exposing it to the Python application. It may be reasonable to produce a warning if tmpnam is called. We cannot replace tempnam with mkstemp for the same reason Posix couldn't: one produces a string, the other one a file handle. What we could do is to expose mkstemp(3) where available. I don't see the value of that, though: it could be done only on systems where mkstemp is available, and we already expose tmpfile(3). In fact, the Linux man page for mkstemp(3) says # Don't use this function, use tmpfile(3) instead. It's # bet­ter defined and more portable. If you still think there is a need for action, please propose a patch. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486434&group_id=5470 From noreply@sourceforge.net Fri Nov 30 12:54:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 04:54:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-487471 ] urllib2 proxyhandle won't work. Message-ID: Bugs item #487471, was opened at 2001-11-30 04:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487471&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Ha Shao (hashao) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2 proxyhandle won't work. Initial Comment: For python 2.2b2: Adding proxyhandler with build_opener, install_opener does not work since order of the handlers does matter (my guess). In the current code, new handlers are appended at the end of the handlers list. At least under windows, the proxyhandle will never got called if it is added with install_opener. HTTPHandler always get called before proxyhandler if the default list is altered. The attached patch try to reserve the order of handlers based on their baseclass when adding new handlers. Handlers with the same baseclasses might be added more than once. Not sure if it is a good thing or bad thing. Current code will raise except when it trys to remove the default handlers twice. Since order does matter, build_opener should be repleased by a list like class such that user can insert their handlers to specific place. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487471&group_id=5470 From noreply@sourceforge.net Fri Nov 30 13:10:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 05:10:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-487471 ] urllib2 proxyhandle won't work. Message-ID: Bugs item #487471, was opened at 2001-11-30 04:54 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487471&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Ha Shao (hashao) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2 proxyhandle won't work. Initial Comment: For python 2.2b2: Adding proxyhandler with build_opener, install_opener does not work since order of the handlers does matter (my guess). In the current code, new handlers are appended at the end of the handlers list. At least under windows, the proxyhandle will never got called if it is added with install_opener. HTTPHandler always get called before proxyhandler if the default list is altered. The attached patch try to reserve the order of handlers based on their baseclass when adding new handlers. Handlers with the same baseclasses might be added more than once. Not sure if it is a good thing or bad thing. Current code will raise except when it trys to remove the default handlers twice. Since order does matter, build_opener should be repleased by a list like class such that user can insert their handlers to specific place. ---------------------------------------------------------------------- >Comment By: Ha Shao (hashao) Date: 2001-11-30 05:10 Message: Logged In: YES user_id=8717 forgot some code and changed a stupid variable name. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487471&group_id=5470 From noreply@sourceforge.net Fri Nov 30 14:15:00 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 06:15:00 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-30 06:15 Message: Logged In: YES user_id=7887 I beg your pardon. Are you saying that there are many changes in Python 2.2 that will blow up code written in Python 2.1!? Why do we have __future__ than!? Why are we concerned about being careful with division upgrade when we have lots of small stuff breaking up *valid* code written one release ago? And, if this was not enough, these changes won't even be documented!?! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 04:21 Message: Logged In: YES user_id=21627 Well, I wouldn't object to anybody documenting this particular change (even though I won't do that myself, either). I'm just pointing out that there are many more changes of this kind, and that it would be an illusion that you could ever produce a complete list. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:36 Message: Logged In: YES user_id=38388 I disagree: any change which could potentially blow up existing programs should at least be mentioned somewhere in the docs, be it the NEWS file, the release page on python.org or (thanks to the great work of Andrew Kuchling) the "What's new in Python x.x" docs. This is simply needed due to the very dynamic way Python works: there's no way to test all paths through a program if you want to port it from one Python version to the next, so we'll have to be very careful about these changes even if they are clearly bug fixes. In the mentioned case, I'd say that the programmer was at fault, though: it's so much easier to test a path for being a directory than to rely on some obscure method return value and also much safer ! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:19 Message: Logged In: YES user_id=21627 Any change is user-visible: for any change, I can construct a program that behaves differently with the change. Otherwise, the change would be useless (except that it may add commentary or some such). In fact, the change *is* documented, namely in the CVS log. Extracting all those fragments into a single document would be time-consuming and pointless: nobody can read through hundreds of changes. Your program would have blown up even if there was such a document. If a change has severe effects on many existing programs, it should be implemented differently. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 15:04:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 07:04:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-487310 ] smtplib mishandles SMTP disconnects Message-ID: Bugs item #487310, was opened at 2001-11-29 16:35 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487310&group_id=5470 Category: Python Library Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Fazal Majid (majid) Assigned to: Nobody/Anonymous (nobody) Summary: smtplib mishandles SMTP disconnects Initial Comment: smtplib handles SMTP disconnects (e.g. timeouts) inconsistently. The smtplib.SMTP.send() method does not reset self.file on a socket error, unlike getreply(), and thus a close() is necessary for in one case but not in the other. Recommendation: have smtplib.SMTP.send() call close() on socket.error just as smtplib.SMTP.getreply() does on EOF on self.file. In the following test case, I have set the SMTP timeout on my local machine to 10 seconds: Python 2.1.1 (#1, Nov 7 2001, 16:18:10) [GCC 2.95.3 20010315 (release)] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import smtplib >>> s = smtplib.SMTP('localhost', 25) >>> import time >>> time.sleep(10) >>> s.sendmail('fmajid@kefta.com', 'fmajid@kefta.com', '...') Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/smtplib.py", line 469, in sendmail (code,resp) = self.helo() File "/usr/local/lib/python2.1/smtplib.py", line 305, in helo self.putcmd("helo", socket.getfqdn()) File "/usr/local/lib/python2.1/smtplib.py", line 249, in putcmd self.send(str) File "/usr/local/lib/python2.1/smtplib.py", line 239, in send raise SMTPServerDisconnected('Server not connected') smtplib.SMTPServerDisconnected: Server not connected >>> s.connect('localhost', 25) Traceback (most recent call last): File "", line 1, in ? File "/usr/local/lib/python2.1/smtplib.py", line 226, in connect (code,msg)=self.getreply() File "/usr/local/lib/python2.1/smtplib.py", line 271, in getreply raise SMTPServerDisconnected("Connection unexpectedly closed") smtplib.SMTPServerDisconnected: Connection unexpectedly closed >>> s.connect('localhost', 25) (220, 'bayazid.kefta.com ESMTP Postfix') ---------------------------------------------------------------------- >Comment By: Fazal Majid (majid) Date: 2001-11-30 07:04 Message: Logged In: YES user_id=110477 I forgot to mention a workaround: if you catch SMTPServerDisconnected, call self.close(). The operation is idempotent so it isn't a problem if you do it twice, for instance if the exception occurs in getreply(). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487310&group_id=5470 From noreply@sourceforge.net Fri Nov 30 16:07:10 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 08:07:10 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Nobody/Anonymous (nobody) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:07 Message: Logged In: YES user_id=21627 No. I think very few of the changes will ever cause problems in real software - including this one. I wouldn't be surprised if this remains the only reported incidence of that change. I *am* saying that for every change that has been made, one could construct an application that breaks under this change. This theoretical potential for breakage shouldn't cause prominent appearance in documentation; everybody who really wants to see documentation for all changes should consult the CVS logs. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-30 06:15 Message: Logged In: YES user_id=7887 I beg your pardon. Are you saying that there are many changes in Python 2.2 that will blow up code written in Python 2.1!? Why do we have __future__ than!? Why are we concerned about being careful with division upgrade when we have lots of small stuff breaking up *valid* code written one release ago? And, if this was not enough, these changes won't even be documented!?! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 04:21 Message: Logged In: YES user_id=21627 Well, I wouldn't object to anybody documenting this particular change (even though I won't do that myself, either). I'm just pointing out that there are many more changes of this kind, and that it would be an illusion that you could ever produce a complete list. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:36 Message: Logged In: YES user_id=38388 I disagree: any change which could potentially blow up existing programs should at least be mentioned somewhere in the docs, be it the NEWS file, the release page on python.org or (thanks to the great work of Andrew Kuchling) the "What's new in Python x.x" docs. This is simply needed due to the very dynamic way Python works: there's no way to test all paths through a program if you want to port it from one Python version to the next, so we'll have to be very careful about these changes even if they are clearly bug fixes. In the mentioned case, I'd say that the programmer was at fault, though: it's so much easier to test a path for being a directory than to rely on some obscure method return value and also much safer ! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:19 Message: Logged In: YES user_id=21627 Any change is user-visible: for any change, I can construct a program that behaves differently with the change. Otherwise, the change would be useless (except that it may add commentary or some such). In fact, the change *is* documented, namely in the CVS log. Extracting all those fragments into a single document would be time-consuming and pointless: nobody can read through hundreds of changes. Your program would have blown up even if there was such a document. If a change has severe effects on many existing programs, it should be implemented differently. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 16:24:45 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 08:24:45 -0800 Subject: [Python-bugs-list] [ python-Bugs-486099 ] socketmodule won't compile on BSDI 4.0 Message-ID: Bugs item #486099, was opened at 2001-11-27 10:19 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: Martin v. Löwis (loewis) Summary: socketmodule won't compile on BSDI 4.0 Initial Comment: Trying to build 2.2b2 on BSDI BSD/OS 4.0 with gcc 2.7.2.1 and gcc 2.95.2, and the socket module won't build. No problems building socket for Python 2.1.1 on this same machine. % ./configure --with-threads=no # for socket(2), without SSL support. _socket socketmodule.c % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from ./Modules/socketmodule.c:187: /usr/local/packages/gcc-2.95.2/lib/gcc-lib/i386-pc-bsdi4.0/2.95.2/include/stddef.h:280: conflicting types for `wint_t' /usr/include/wchar.h:7: previous declaration of `wint_t' In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c: In function `gai_strerror': Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use in this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use in this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use in this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use in this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use in this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- >Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-30 08:24 Message: Logged In: YES user_id=85984 % gcc -o aicheck2 aicheck2.c % ./aicheck2 fail 11 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:35 Message: Logged In: YES user_id=21627 Arrghh. I was trying to find out which of the tests fail, but attached the wrong version of the file: I meant to put printfs into each failure. Please try aicheck2.c instead. Anyway, it seems that your system is indeed one of those where the test should find out problems with the C library (unless the test that fails is bogus, which we can investigate once we know what it is that fails). I'll try to come up with a patch that allows usage of the getaddrinfo emulation even on systems where getaddrinfo is in the C library. ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-29 18:18 Message: Logged In: YES user_id=85984 aicheck.c compiled without problem on my system. % gcc -o aicheck aicheck.c % Running the resulting binary produces no output. I'm not exactly sure what kind of output you are looking for (taking a guess): % gdb /home/jason/temp/aicheck GDB is free software and you are welcome to distribute copies of it under certain conditions; type "show copying" to see the conditions. There is absolutely no warranty for GDB; type "show warranty" for details. GDB 4.16 (i386-unknown-bsdi4.0), Copyright 1996 Free Software Foundation, Inc...(no debugging symbols found)... (gdb) run Starting program: /home/jason/temp/aicheck (no debugging symbols found)...(no debugging symbols found)...(no debugging symbols found)... Program exited with code 01. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:51 Message: Logged In: YES user_id=21627 Thansk for the files. I now trying to figure out why Python refuses to use the getaddrinfo implementation in your C library. It may be that it is really broken, in which case we must make getaddrinfo.c work correctly. Perhaps it ought to work,and configure just fails to correctly see that. So please compile the attached file aicheck.c on your system, and report output and status code (looking at $?); if it fails to compile, report the compile errors. ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-28 12:21 Message: Logged In: YES user_id=85984 Attached are three files: - pyconfig.h from a fresh gcc 2.7.2.1 build - config.log from a fresh gcc 2.7.2.1 build - /usr/include/netdb.h - the getaddrinfo manual page from my system Indeed, EAI_ADDRFAMILY is defined in /usr/include/netdb.h, but EAI_MAX is not. Let me know if I can provide any more information. Thanks. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-28 01:30 Message: Logged In: YES user_id=21627 Ok, then we need to investigate the configure results. Can you please attach pyconfig.h and config.log to this report? Also, can you please confirm that you build Python from scratch with 2.7.2.1 (i.e untarring a fresh source tree, to avoid using a config.cache left over from gcc 2.95.2)? Furthermore, can you please report details of the getaddrinfo implementation on your system? Does it provide one? In the C library? If not, can you confirm that EAI_ADDRFAMILY is defined in your system headers, but EAI_MAX is not? If yes, can you speculate as to why configure thinks your system does not have getaddrinfo? ---------------------------------------------------------------------- Comment By: Jason R. Mastaler (jasonrm) Date: 2001-11-27 15:21 Message: Logged In: YES user_id=85984 I probably should have included the output from gcc 2.7.2.1 instead which I know is properly installed. The results are similar, although there is no mention of `wint_t': % gmake gcc -D_HAVE_BSDI -g -O3 -Wall -Wstrict-prototypes -I. -I./Include -DHAVE_CONFIG_H -c ./Modules/socketmodule.c -o Modules/socketmodule.o In file included from Include/pyport.h:93, from Include/Python.h:60, from ./Modules/socketmodule.c:78: /usr/include/math.h:177: warning: function declaration isn't a prototype /usr/include/math.h:257: warning: function declaration isn't a prototype In file included from Include/unicodeobject.h:118, from Include/Python.h:69, from ./Modules/socketmodule.c:78: /usr/include/wchar.h:15: warning: function declaration isn't a prototype In file included from /usr/include/signal.h:46, from ./Modules/socketmodule.c:140: /usr/include/sys/signal.h:121: warning: function declaration isn't a prototype /usr/include/sys/signal.h:164: warning: function declaration isn't a prototype Modules/getaddrinfo.c: In function `gai_strerror': In file included from ./Modules/socketmodule.c:236: Modules/getaddrinfo.c:205: `EAI_MAX' undeclared (first use this function) Modules/getaddrinfo.c:205: (Each undeclared identifier is reported only once Modules/getaddrinfo.c:205: for each function it appears in.) Modules/getaddrinfo.c: In function `getaddrinfo': Modules/getaddrinfo.c:285: `EAI_BADHINTS' undeclared (first use this function) Modules/getaddrinfo.c:286: `AI_MASK' undeclared (first use this function) Modules/getaddrinfo.c:376: `EAI_PROTOCOL' undeclared (first use this function) Modules/getaddrinfo.c:464: `AI_NUMERICHOST' undeclared (first use this function) ./Modules/socketmodule.c: In function `PySocketSock_init': ./Modules/socketmodule.c:1818: warning: function declaration isn't a prototype ./Modules/socketmodule.c: In function `PySocket_fromfd': ./Modules/socketmodule.c:2304: warning: function declaration isn't a prototype gmake: *** [Modules/socketmodule.o] Error 1 ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-27 15:08 Message: Logged In: YES user_id=21627 Are you sure the compiler is properly installed? That you have wint_t definition both in wchar.h and stddef.h (as fixincluded by gcc) is not Python's doing; either the header files of your operating system are corrupt, or the compiler is broken. If you cannot figure out the details, please attach both header files to this report. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-27 12:07 Message: Logged In: YES user_id=31435 Assigned to Martin. ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=486099&group_id=5470 From noreply@sourceforge.net Fri Nov 30 16:23:59 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 08:23:59 -0800 Subject: [Python-bugs-list] [ python-Bugs-487321 ] Installation fails on module-xmllib.html Message-ID: Bugs item #487321, was opened at 2001-11-29 17:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: Installation fails on module-xmllib.html Initial Comment: On NT 4.0 SP6 installation of Python 2.2b2 hangs with the following message: "Copying Python Documentation (HTML); F:\Python22\Doc\lib\module-xmllib.html Upon further investigation it appears the installation program is getting hung up while it creates an infinite length .tmp file (multiple Gb) that consumes all available disk space on the installation drive. The installation file Python-2.2b2.exe has a file length of 7,008,533 bytes. I've now tried to install both versions 2.1 and 2.2b2 - both installations have failed for different reasons. Please get the Windows installations fixed! ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:23 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of this file as well? cf59bb416bc43f6bf77a3329db901f6f Python-2.2b2.exe ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:27 Message: Logged In: YES user_id=31435 See comment on your bug 487317: you're unique here too. Can you reproduce on a physically distinct machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 From noreply@sourceforge.net Fri Nov 30 16:19:55 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 08:19:55 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- >Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:19 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of the file you've downloaded? It is 39ef54d3e29ea402c8b693b4f3fedd4a Python-2.1.1.exe You can get an md5sum executable from http://etree.org/md5com.html ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:25 Message: Logged In: YES user_id=31435 Tens of thousands of Windows users have run this installer without problems, so-- and especially given your other bug report about 2.2b2 --the presumption has to be that something is uniquely screwed up on the specific box you tried it on. Can you reproduce on another (physically distinct) machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 From noreply@sourceforge.net Fri Nov 30 16:30:15 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 08:30:15 -0800 Subject: [Python-bugs-list] [ python-Bugs-487165 ] difficult to find string interpolation Message-ID: Bugs item #487165, was opened at 2001-11-29 10:48 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487165&group_id=5470 Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Gábor BORGULYA (borgulya) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: difficult to find string interpolation Initial Comment: I tried to look up the possibilities how to formulate expressions like 'Hello %s\n' % name After a long search I found out that this is "string interpolation". But searching for the word 'interpolation' resulted only in pages about interpolation errors. Finally I found the page: Python library reference, 2.1.5.2 String Formatting Operations. This is what I wanted to see. This page does not contain the word "interpolation". I think this part of the documentation should be made easier to find. ---------------------------------------------------------------------- >Comment By: Skip Montanaro (montanaro) Date: 2001-11-30 08:30 Message: Logged In: YES user_id=44345 I just tried adding the following extra index entries to the string formatting section in libstdtypes.tex: \index{\% formatting} \index{string interpolation} \index{string!interpolation} \index{interpolation, string} All but the first one worked. The first sort of worked, but the "% formatting" entry came up in the index under the character "<" for some reason. Perhaps Fred knows how to work around that problem. Skip ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487165&group_id=5470 From noreply@sourceforge.net Fri Nov 30 17:43:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 09:43:28 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-30 09:43 Message: Logged In: YES user_id=31435 Thanks for finding that program, Martin! This should be received much better than my usual advice to compute MD5 with md5sum.py . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:19 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of the file you've downloaded? It is 39ef54d3e29ea402c8b693b4f3fedd4a Python-2.1.1.exe You can get an md5sum executable from http://etree.org/md5com.html ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:25 Message: Logged In: YES user_id=31435 Tens of thousands of Windows users have run this installer without problems, so-- and especially given your other bug report about 2.2b2 --the presumption has to be that something is uniquely screwed up on the specific box you tried it on. Can you reproduce on another (physically distinct) machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 From noreply@sourceforge.net Fri Nov 30 19:01:06 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 11:01:06 -0800 Subject: [Python-bugs-list] [ python-Bugs-487331 ] time mod's timezone doesn't honor TZ var Message-ID: Bugs item #487331, was opened at 2001-11-29 18:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None >Priority: 3 Submitted By: Jason R. Mastaler (jasonrm) >Assigned to: M.-A. Lemburg (lemburg) Summary: time mod's timezone doesn't honor TZ var Initial Comment: Python's time module seems to get the timezone name when it's first imported, but it doesn't change it if the TZ environment variable is later set. On the other hand, the time of day DOES change accordingly. Thus, setting TZ after the time module is imported results in a completely broken rfc2822 date - time in the new TZ, but with a bogus UTC offset and timezone (neither the old nor new). Here is the problem illustrated. You will see that after setting TZ, altzone, timezone, and tzname don't change, while asctime does. $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import time,os >>> print time.asctime() Thu Nov 29 18:47:39 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print time.asctime() Fri Nov 30 14:47:52 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') Here is an example of why this is a problem in practice. One of my applications sets the TZ environment variable based on a "TIMEZONE" setting in the user's configuration file. It later uses the `email' module to create a "Date:" header for outgoing mail messages. Because of this bug, the resulting "Date:" is bogus. e.g, $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import email,os >>> print email.Utils.formatdate(localtime=1) Thu, 29 Nov 2001 18:52:34 -0700 >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print email.Utils.formatdate(localtime=1) Fri, 30 Nov 2001 14:52:41 -0600 The `-0600' should be `+1300', and `-0600' isn't correct even for the original timezone! You might say: "Work around this by setting TZ before the time module is first imported." -- No, that is too fragile, and too difficult to do with complex applications. Many other modules in turn import time, etc. I suggest instead that the TZ environment variable be consulted each time the timezone is accessed. ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 11:01 Message: Logged In: YES user_id=38388 The time module is a very thin layer on top of the C lib time APIs. There's nothing much Python can do about errors in those APIs, e.g. not recognizing changes to TZ. Also note that the tzset() API which inits the time APIs w/r to the TZ environment variable is only called at time module import time. To expose the dynamic changes you are looking for, it would have to call tzset() prior to all calls to asctime() et al. -- much too time consuming ! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 From noreply@sourceforge.net Fri Nov 30 20:11:23 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 12:11:23 -0800 Subject: [Python-bugs-list] [ python-Bugs-487331 ] time mod's timezone doesn't honor TZ var Message-ID: Bugs item #487331, was opened at 2001-11-29 18:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 3 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: M.-A. Lemburg (lemburg) Summary: time mod's timezone doesn't honor TZ var Initial Comment: Python's time module seems to get the timezone name when it's first imported, but it doesn't change it if the TZ environment variable is later set. On the other hand, the time of day DOES change accordingly. Thus, setting TZ after the time module is imported results in a completely broken rfc2822 date - time in the new TZ, but with a bogus UTC offset and timezone (neither the old nor new). Here is the problem illustrated. You will see that after setting TZ, altzone, timezone, and tzname don't change, while asctime does. $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import time,os >>> print time.asctime() Thu Nov 29 18:47:39 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print time.asctime() Fri Nov 30 14:47:52 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') Here is an example of why this is a problem in practice. One of my applications sets the TZ environment variable based on a "TIMEZONE" setting in the user's configuration file. It later uses the `email' module to create a "Date:" header for outgoing mail messages. Because of this bug, the resulting "Date:" is bogus. e.g, $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import email,os >>> print email.Utils.formatdate(localtime=1) Thu, 29 Nov 2001 18:52:34 -0700 >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print email.Utils.formatdate(localtime=1) Fri, 30 Nov 2001 14:52:41 -0600 The `-0600' should be `+1300', and `-0600' isn't correct even for the original timezone! You might say: "Work around this by setting TZ before the time module is first imported." -- No, that is too fragile, and too difficult to do with complex applications. Many other modules in turn import time, etc. I suggest instead that the TZ environment variable be consulted each time the timezone is accessed. ---------------------------------------------------------------------- >Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-30 12:11 Message: Logged In: YES user_id=12800 Perhaps we should expose tzset() to Python? We'd also have to reset timezone, altzone, daylight, and tzname in the module after calling tzset(). It's a bit of work, and adds a feature so it can't go into Python 2.2, but if you think this is a viable approach, re-assign this bug to me and I'll deal with it for Python 2.3. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 11:01 Message: Logged In: YES user_id=38388 The time module is a very thin layer on top of the C lib time APIs. There's nothing much Python can do about errors in those APIs, e.g. not recognizing changes to TZ. Also note that the tzset() API which inits the time APIs w/r to the TZ environment variable is only called at time module import time. To expose the dynamic changes you are looking for, it would have to call tzset() prior to all calls to asctime() et al. -- much too time consuming ! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 From noreply@sourceforge.net Fri Nov 30 20:53:47 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 12:53:47 -0800 Subject: [Python-bugs-list] [ python-Bugs-487331 ] time mod's timezone doesn't honor TZ var Message-ID: Bugs item #487331, was opened at 2001-11-29 18:09 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 Category: Python Library Group: Python 2.2 Status: Open Resolution: None Priority: 3 Submitted By: Jason R. Mastaler (jasonrm) Assigned to: M.-A. Lemburg (lemburg) Summary: time mod's timezone doesn't honor TZ var Initial Comment: Python's time module seems to get the timezone name when it's first imported, but it doesn't change it if the TZ environment variable is later set. On the other hand, the time of day DOES change accordingly. Thus, setting TZ after the time module is imported results in a completely broken rfc2822 date - time in the new TZ, but with a bogus UTC offset and timezone (neither the old nor new). Here is the problem illustrated. You will see that after setting TZ, altzone, timezone, and tzname don't change, while asctime does. $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import time,os >>> print time.asctime() Thu Nov 29 18:47:39 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print time.asctime() Fri Nov 30 14:47:52 2001 >>> print time.altzone,time.timezone,time.tzname 21600 25200 ('MST', 'MDT') Here is an example of why this is a problem in practice. One of my applications sets the TZ environment variable based on a "TIMEZONE" setting in the user's configuration file. It later uses the `email' module to create a "Date:" header for outgoing mail messages. Because of this bug, the resulting "Date:" is bogus. e.g, $ python2.2 Python 2.2b2 (#1, Nov 18 2001, 18:20:32) [GCC 2.95.3 20010315 (release) [FreeBSD]] on freebsd4 Type "help", "copyright", "credits" or "license" for more information. >>> import email,os >>> print email.Utils.formatdate(localtime=1) Thu, 29 Nov 2001 18:52:34 -0700 >>> >>> os.environ['TZ'] = "Pacific/Auckland" >>> print email.Utils.formatdate(localtime=1) Fri, 30 Nov 2001 14:52:41 -0600 The `-0600' should be `+1300', and `-0600' isn't correct even for the original timezone! You might say: "Work around this by setting TZ before the time module is first imported." -- No, that is too fragile, and too difficult to do with complex applications. Many other modules in turn import time, etc. I suggest instead that the TZ environment variable be consulted each time the timezone is accessed. ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 12:53 Message: Logged In: YES user_id=38388 I don't know whether this is such a good idea -- after all it's fairly easy to manage timezones in the application rather than trying to fiddle the C libs routines into producing the output you intend. Also note that tzset() is *not* thread safe. BTW, Jason, you may want to have a look at mxDateTime -- perhaps it already has what you are looking for. ---------------------------------------------------------------------- Comment By: Barry Warsaw (bwarsaw) Date: 2001-11-30 12:11 Message: Logged In: YES user_id=12800 Perhaps we should expose tzset() to Python? We'd also have to reset timezone, altzone, daylight, and tzname in the module after calling tzset(). It's a bit of work, and adds a feature so it can't go into Python 2.2, but if you think this is a viable approach, re-assign this bug to me and I'll deal with it for Python 2.3. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 11:01 Message: Logged In: YES user_id=38388 The time module is a very thin layer on top of the C lib time APIs. There's nothing much Python can do about errors in those APIs, e.g. not recognizing changes to TZ. Also note that the tzset() API which inits the time APIs w/r to the TZ environment variable is only called at time module import time. To expose the dynamic changes you are looking for, it would have to call tzset() prior to all calls to asctime() et al. -- much too time consuming ! ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487331&group_id=5470 From noreply@sourceforge.net Fri Nov 30 22:26:33 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 14:26:33 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 >Status: Closed Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) >Assigned to: Tim Peters (tim_one) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-30 14:26 Message: Logged In: YES user_id=31435 Does anyone know why opening a directory for reading doesn't complain on Linux? It has always complained on Windows. If readline() is doomed to fail, why do we allow opening a directory for reading at all? OTOH, if it's not insane to open a directory for reading on Linux, why does readline() complain on Linux? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:07 Message: Logged In: YES user_id=21627 No. I think very few of the changes will ever cause problems in real software - including this one. I wouldn't be surprised if this remains the only reported incidence of that change. I *am* saying that for every change that has been made, one could construct an application that breaks under this change. This theoretical potential for breakage shouldn't cause prominent appearance in documentation; everybody who really wants to see documentation for all changes should consult the CVS logs. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-30 06:15 Message: Logged In: YES user_id=7887 I beg your pardon. Are you saying that there are many changes in Python 2.2 that will blow up code written in Python 2.1!? Why do we have __future__ than!? Why are we concerned about being careful with division upgrade when we have lots of small stuff breaking up *valid* code written one release ago? And, if this was not enough, these changes won't even be documented!?! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 04:21 Message: Logged In: YES user_id=21627 Well, I wouldn't object to anybody documenting this particular change (even though I won't do that myself, either). I'm just pointing out that there are many more changes of this kind, and that it would be an illusion that you could ever produce a complete list. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:36 Message: Logged In: YES user_id=38388 I disagree: any change which could potentially blow up existing programs should at least be mentioned somewhere in the docs, be it the NEWS file, the release page on python.org or (thanks to the great work of Andrew Kuchling) the "What's new in Python x.x" docs. This is simply needed due to the very dynamic way Python works: there's no way to test all paths through a program if you want to port it from one Python version to the next, so we'll have to be very careful about these changes even if they are clearly bug fixes. In the mentioned case, I'd say that the programmer was at fault, though: it's so much easier to test a path for being a directory than to rely on some obscure method return value and also much safer ! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:19 Message: Logged In: YES user_id=21627 Any change is user-visible: for any change, I can construct a program that behaves differently with the change. Otherwise, the change would be useless (except that it may add commentary or some such). In fact, the change *is* documented, namely in the CVS log. Extracting all those fragments into a single document would be time-consuming and pointless: nobody can read through hundreds of changes. Your program would have blown up even if there was such a document. If a change has severe effects on many existing programs, it should be implemented differently. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 22:34:51 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 14:34:51 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- >Comment By: Barry Kumnick (barryk) Date: 2001-11-30 14:34 Message: Logged In: YES user_id=389562 Hi, Thanks for getting back to me. First I have a correction to make: it was the tk83.dll not the tcl83.dll that was reported as corrupt. Actually, I am running NT 4 Enterprise Server SP6 on a dual processor PIII box. I tried to install on another box running NT 4 Enterprise Server SP5 with a single processor, and got the same error. However, on that machine I was able to keep pressing retry, and the progress bar kept advancing. In fact it went up a few percent every time I pressed the retry button on the error dialog. It continued to advance past 100%. I stopped trying after it got to around 113% and it said the system needed to be rebooted to complete the installation. I did that. Apparently, everything installed except the tk83.dll. I also tried the md5sum.exe application Martin suggested, but it reports: md5sum: Python-2.1.1.exe: no properly formated MD5 checksum lines found. Do you have another suggestion for computing the MD5 checksum? I don't have md5sum.py on my system. BTW: With the current 2.1.1 installation, I was able to get the Python command shell to execute, but when attempting to launch IDLE, it fails with an error message saying the tk83.dll can not be found. Is there somewhere I can download the tk83.dll separately as a workaround? Thanks, Barry Kumnick ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-30 09:43 Message: Logged In: YES user_id=31435 Thanks for finding that program, Martin! This should be received much better than my usual advice to compute MD5 with md5sum.py . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:19 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of the file you've downloaded? It is 39ef54d3e29ea402c8b693b4f3fedd4a Python-2.1.1.exe You can get an md5sum executable from http://etree.org/md5com.html ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:25 Message: Logged In: YES user_id=31435 Tens of thousands of Windows users have run this installer without problems, so-- and especially given your other bug report about 2.2b2 --the presumption has to be that something is uniquely screwed up on the specific box you tried it on. Can you reproduce on another (physically distinct) machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 From noreply@sourceforge.net Fri Nov 30 22:39:26 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 14:39:26 -0800 Subject: [Python-bugs-list] [ python-Bugs-487703 ] Replace strcat, strcpy Message-ID: Bugs item #487703, was opened at 2001-11-30 14:39 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487703&group_id=5470 Category: Python Interpreter Core Group: None Status: Open Resolution: None Priority: 5 Submitted By: Tim Peters (tim_one) Assigned to: Nobody/Anonymous (nobody) Summary: Replace strcat, strcpy Initial Comment: Alex Martelli sent this URL for a paper describing the safer strlcat and strlcpy functions: >http://www.usenix.org/events/usenix99/full_papers/mill ert/millert_html/> ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487703&group_id=5470 From noreply@sourceforge.net Fri Nov 30 22:42:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 14:42:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-487321 ] Installation fails on module-xmllib.html Message-ID: Bugs item #487321, was opened at 2001-11-29 17:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: Installation fails on module-xmllib.html Initial Comment: On NT 4.0 SP6 installation of Python 2.2b2 hangs with the following message: "Copying Python Documentation (HTML); F:\Python22\Doc\lib\module-xmllib.html Upon further investigation it appears the installation program is getting hung up while it creates an infinite length .tmp file (multiple Gb) that consumes all available disk space on the installation drive. The installation file Python-2.2b2.exe has a file length of 7,008,533 bytes. I've now tried to install both versions 2.1 and 2.2b2 - both installations have failed for different reasons. Please get the Windows installations fixed! ---------------------------------------------------------------------- >Comment By: Barry Kumnick (barryk) Date: 2001-11-30 14:42 Message: Logged In: YES user_id=389562 Hi Tim, I have reproduced the same problem on a physically distinct machine per your suggestion. The first machine is an NT4 Enterprise Server Dual Processor Dual Boot PIII box with SP6. The second machine is an NT4 Enterprise Server Single Processor Dual Boot PIII box with SP5. The installations failed on both machines with the symptoms noted in my previous post. Can you advise me how to verify the md5sum on the download file? I downloaded the md5sum.exe program Martin suggested in my other bug report, but it returns a message saying: md5sum: Python-2.2b2.exe: no properly formated MD5 checksum lines found. Also, I don't have the md5sum.py module. I might be able to use that from the command line if you could tell me where to download it. Thanks, Barry Kumnick ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:23 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of this file as well? cf59bb416bc43f6bf77a3329db901f6f Python-2.2b2.exe ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:27 Message: Logged In: YES user_id=31435 See comment on your bug 487317: you're unique here too. Can you reproduce on a physically distinct machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 From noreply@sourceforge.net Fri Nov 30 22:44:30 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 14:44:30 -0800 Subject: [Python-bugs-list] [ python-Bugs-487277 ] Different behavior of readline() Message-ID: Bugs item #487277, was opened at 2001-11-29 14:43 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 Category: Python Interpreter Core Group: Python 2.2 Status: Closed Resolution: None Priority: 5 Submitted By: Gustavo Niemeyer (niemeyer) Assigned to: Tim Peters (tim_one) Summary: Different behavior of readline() Initial Comment: The behavior of readline() has changed on 2.2. This should be documented. Example: Python 2.1 (#1, Jun 22 2001, 17:13:13) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-i386 Type "copyright", "credits" or "license" for more information. >>> open("/etc").readline() '' Python 2.2b2+ (#1, Nov 27 2001, 21:39:35) [GCC 2.95.3 20010315 (release) (conectiva)] on linux-ppc Type "help", "copyright", "credits" or "license" for more information. >>> open("/etc").readline() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory ---------------------------------------------------------------------- >Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 14:44 Message: Logged In: YES user_id=38388 For the record, this is what I get with Python 2.2b2 on Linux: >>> f = open('/home/lemburg/') >>> f.read() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory >>> f.readline() '' >>> f.readlines() Traceback (most recent call last): File "", line 1, in ? IOError: [Errno 21] Is a directory >>> Reading the man-page for open(), it seems that directories play some role (though it's not clear which...): open() options: ... O_DIRECTORY If pathname is not a directory, cause the open to fail. This flag is Linux-specific, and was added in kernel version 2.1.126, to avoid denial-of-ser­ vice problems if opendir(3) is called on a FIFO or tape device, but should not be used outside of the implementation of opendir. errno values: ... EISDIR pathname refers to a directory and the access requested involved writing. EACCES The requested access to the file is not allowed, or one of the directories in pathname did not allow search (execute) permission, or the file did not exist yet and write access to the parent directory is not allowed. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-30 14:26 Message: Logged In: YES user_id=31435 Does anyone know why opening a directory for reading doesn't complain on Linux? It has always complained on Windows. If readline() is doomed to fail, why do we allow opening a directory for reading at all? OTOH, if it's not insane to open a directory for reading on Linux, why does readline() complain on Linux? ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:07 Message: Logged In: YES user_id=21627 No. I think very few of the changes will ever cause problems in real software - including this one. I wouldn't be surprised if this remains the only reported incidence of that change. I *am* saying that for every change that has been made, one could construct an application that breaks under this change. This theoretical potential for breakage shouldn't cause prominent appearance in documentation; everybody who really wants to see documentation for all changes should consult the CVS logs. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-30 06:15 Message: Logged In: YES user_id=7887 I beg your pardon. Are you saying that there are many changes in Python 2.2 that will blow up code written in Python 2.1!? Why do we have __future__ than!? Why are we concerned about being careful with division upgrade when we have lots of small stuff breaking up *valid* code written one release ago? And, if this was not enough, these changes won't even be documented!?! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 04:21 Message: Logged In: YES user_id=21627 Well, I wouldn't object to anybody documenting this particular change (even though I won't do that myself, either). I'm just pointing out that there are many more changes of this kind, and that it would be an illusion that you could ever produce a complete list. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2001-11-30 01:36 Message: Logged In: YES user_id=38388 I disagree: any change which could potentially blow up existing programs should at least be mentioned somewhere in the docs, be it the NEWS file, the release page on python.org or (thanks to the great work of Andrew Kuchling) the "What's new in Python x.x" docs. This is simply needed due to the very dynamic way Python works: there's no way to test all paths through a program if you want to port it from one Python version to the next, so we'll have to be very careful about these changes even if they are clearly bug fixes. In the mentioned case, I'd say that the programmer was at fault, though: it's so much easier to test a path for being a directory than to rely on some obscure method return value and also much safer ! ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 01:19 Message: Logged In: YES user_id=21627 Any change is user-visible: for any change, I can construct a program that behaves differently with the change. Otherwise, the change would be useless (except that it may add commentary or some such). In fact, the change *is* documented, namely in the CVS log. Extracting all those fragments into a single document would be time-consuming and pointless: nobody can read through hundreds of changes. Your program would have blown up even if there was such a document. If a change has severe effects on many existing programs, it should be implemented differently. ---------------------------------------------------------------------- Comment By: Gustavo Niemeyer (niemeyer) Date: 2001-11-29 16:10 Message: Logged In: YES user_id=7887 I think *any* change visible at user space should be documented. I discovered this change because a program I have never read in my life (in fact, I didn't even know it was written in python) has blown up in my hands. It was reading files and safely ignoring directories by testing readline()'s output. ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-29 15:53 Message: Logged In: YES user_id=21627 I don't think this needs to be documented. The change is a bug fix, reading from a directory was never supposed to work (whether in lines or not). ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487277&group_id=5470 From noreply@sourceforge.net Fri Nov 30 22:54:29 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 14:54:29 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-30 14:54 Message: Logged In: YES user_id=31435 It sounds like you were trying to run md5sum with its "-c" switch: don't. Do it like so: C:\Code>\updates\md5sum Python-2.1.1.exe 39ef54d3e29ea402c8b693b4f3fedd4a *Python-2.1.1.exe C:\Code> There are no corrupt files in the installers we ship, so stop trying to pursue that path: you'll just get in worse trouble if you do. Something else is hosed on your system. BTW, you can open the installer (as a file!) in WinZip, and use WinZip's Action -> Test to test for corruption. md5 is a more powerful whole-file check, but the installer saves a CRC code for each individual file and that's what the WinZip Test looks at. If either md5 or WinZip complains, your installer got corrupted during (or maybe even after) downloading. ---------------------------------------------------------------------- Comment By: Barry Kumnick (barryk) Date: 2001-11-30 14:34 Message: Logged In: YES user_id=389562 Hi, Thanks for getting back to me. First I have a correction to make: it was the tk83.dll not the tcl83.dll that was reported as corrupt. Actually, I am running NT 4 Enterprise Server SP6 on a dual processor PIII box. I tried to install on another box running NT 4 Enterprise Server SP5 with a single processor, and got the same error. However, on that machine I was able to keep pressing retry, and the progress bar kept advancing. In fact it went up a few percent every time I pressed the retry button on the error dialog. It continued to advance past 100%. I stopped trying after it got to around 113% and it said the system needed to be rebooted to complete the installation. I did that. Apparently, everything installed except the tk83.dll. I also tried the md5sum.exe application Martin suggested, but it reports: md5sum: Python-2.1.1.exe: no properly formated MD5 checksum lines found. Do you have another suggestion for computing the MD5 checksum? I don't have md5sum.py on my system. BTW: With the current 2.1.1 installation, I was able to get the Python command shell to execute, but when attempting to launch IDLE, it fails with an error message saying the tk83.dll can not be found. Is there somewhere I can download the tk83.dll separately as a workaround? Thanks, Barry Kumnick ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-30 09:43 Message: Logged In: YES user_id=31435 Thanks for finding that program, Martin! This should be received much better than my usual advice to compute MD5 with md5sum.py . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:19 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of the file you've downloaded? It is 39ef54d3e29ea402c8b693b4f3fedd4a Python-2.1.1.exe You can get an md5sum executable from http://etree.org/md5com.html ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:25 Message: Logged In: YES user_id=31435 Tens of thousands of Windows users have run this installer without problems, so-- and especially given your other bug report about 2.2b2 --the presumption has to be that something is uniquely screwed up on the specific box you tried it on. Can you reproduce on another (physically distinct) machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 From noreply@sourceforge.net Fri Nov 30 23:11:28 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 15:11:28 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- >Comment By: Barry Kumnick (barryk) Date: 2001-11-30 15:11 Message: Logged In: YES user_id=389562 I got the md5sum to work. It is reporting a different checksum than the one you provided, so the downloaded file must have been corrupted. I'll try to download a new copy. Thanks for your help. Barry Kumnick ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-30 14:54 Message: Logged In: YES user_id=31435 It sounds like you were trying to run md5sum with its "-c" switch: don't. Do it like so: C:\Code>\updates\md5sum Python-2.1.1.exe 39ef54d3e29ea402c8b693b4f3fedd4a *Python-2.1.1.exe C:\Code> There are no corrupt files in the installers we ship, so stop trying to pursue that path: you'll just get in worse trouble if you do. Something else is hosed on your system. BTW, you can open the installer (as a file!) in WinZip, and use WinZip's Action -> Test to test for corruption. md5 is a more powerful whole-file check, but the installer saves a CRC code for each individual file and that's what the WinZip Test looks at. If either md5 or WinZip complains, your installer got corrupted during (or maybe even after) downloading. ---------------------------------------------------------------------- Comment By: Barry Kumnick (barryk) Date: 2001-11-30 14:34 Message: Logged In: YES user_id=389562 Hi, Thanks for getting back to me. First I have a correction to make: it was the tk83.dll not the tcl83.dll that was reported as corrupt. Actually, I am running NT 4 Enterprise Server SP6 on a dual processor PIII box. I tried to install on another box running NT 4 Enterprise Server SP5 with a single processor, and got the same error. However, on that machine I was able to keep pressing retry, and the progress bar kept advancing. In fact it went up a few percent every time I pressed the retry button on the error dialog. It continued to advance past 100%. I stopped trying after it got to around 113% and it said the system needed to be rebooted to complete the installation. I did that. Apparently, everything installed except the tk83.dll. I also tried the md5sum.exe application Martin suggested, but it reports: md5sum: Python-2.1.1.exe: no properly formated MD5 checksum lines found. Do you have another suggestion for computing the MD5 checksum? I don't have md5sum.py on my system. BTW: With the current 2.1.1 installation, I was able to get the Python command shell to execute, but when attempting to launch IDLE, it fails with an error message saying the tk83.dll can not be found. Is there somewhere I can download the tk83.dll separately as a workaround? Thanks, Barry Kumnick ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-30 09:43 Message: Logged In: YES user_id=31435 Thanks for finding that program, Martin! This should be received much better than my usual advice to compute MD5 with md5sum.py . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:19 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of the file you've downloaded? It is 39ef54d3e29ea402c8b693b4f3fedd4a Python-2.1.1.exe You can get an md5sum executable from http://etree.org/md5com.html ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:25 Message: Logged In: YES user_id=31435 Tens of thousands of Windows users have run this installer without problems, so-- and especially given your other bug report about 2.2b2 --the presumption has to be that something is uniquely screwed up on the specific box you tried it on. Can you reproduce on another (physically distinct) machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 From noreply@sourceforge.net Fri Nov 30 23:15:39 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 15:15:39 -0800 Subject: [Python-bugs-list] [ python-Bugs-487321 ] Installation fails on module-xmllib.html Message-ID: Bugs item #487321, was opened at 2001-11-29 17:33 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 Category: Installation Group: Python 2.2 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: Installation fails on module-xmllib.html Initial Comment: On NT 4.0 SP6 installation of Python 2.2b2 hangs with the following message: "Copying Python Documentation (HTML); F:\Python22\Doc\lib\module-xmllib.html Upon further investigation it appears the installation program is getting hung up while it creates an infinite length .tmp file (multiple Gb) that consumes all available disk space on the installation drive. The installation file Python-2.2b2.exe has a file length of 7,008,533 bytes. I've now tried to install both versions 2.1 and 2.2b2 - both installations have failed for different reasons. Please get the Windows installations fixed! ---------------------------------------------------------------------- >Comment By: Barry Kumnick (barryk) Date: 2001-11-30 15:15 Message: Logged In: YES user_id=389562 I just generated the md5 sum on Python-2.2b2.exe. It is also coming back bad. I'll have to try downloading again. Thanks, Barry Kumnick ---------------------------------------------------------------------- Comment By: Barry Kumnick (barryk) Date: 2001-11-30 14:42 Message: Logged In: YES user_id=389562 Hi Tim, I have reproduced the same problem on a physically distinct machine per your suggestion. The first machine is an NT4 Enterprise Server Dual Processor Dual Boot PIII box with SP6. The second machine is an NT4 Enterprise Server Single Processor Dual Boot PIII box with SP5. The installations failed on both machines with the symptoms noted in my previous post. Can you advise me how to verify the md5sum on the download file? I downloaded the md5sum.exe program Martin suggested in my other bug report, but it returns a message saying: md5sum: Python-2.2b2.exe: no properly formated MD5 checksum lines found. Also, I don't have the md5sum.py module. I might be able to use that from the command line if you could tell me where to download it. Thanks, Barry Kumnick ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:23 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of this file as well? cf59bb416bc43f6bf77a3329db901f6f Python-2.2b2.exe ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:27 Message: Logged In: YES user_id=31435 See comment on your bug 487317: you're unique here too. Can you reproduce on a physically distinct machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487321&group_id=5470 From noreply@sourceforge.net Fri Nov 30 23:50:24 2001 From: noreply@sourceforge.net (noreply@sourceforge.net) Date: Fri, 30 Nov 2001 15:50:24 -0800 Subject: [Python-bugs-list] [ python-Bugs-487317 ] tcl83.dll is corrupt Message-ID: Bugs item #487317, was opened at 2001-11-29 17:11 You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470 Category: Installation Group: Python 2.1.1 Status: Open Resolution: None Priority: 5 Submitted By: Barry Kumnick (barryk) Assigned to: Tim Peters (tim_one) Summary: tcl83.dll is corrupt Initial Comment: On NT4 SP6, installation of Python-2.1.1.exe fails with an error message saying tcl83.dll is corrupt. The downloaded file size is correct - 6,310,277 bytes. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2001-11-30 15:50 Message: Logged In: YES user_id=31435 Barry, from exactly where are you downloading the installer? SourceForge? python.org? If the latter, via HTTP or via FTP? Which program are you using to download? It's remotely possible that one of the master copies recently got corrupted, and if you tell me where you're getting it from, I'd be happy to try downloading it myself too. I hate to mention it, but some corporations are known to run firewalls that systematically corrupt binary downloads. ---------------------------------------------------------------------- Comment By: Barry Kumnick (barryk) Date: 2001-11-30 15:11 Message: Logged In: YES user_id=389562 I got the md5sum to work. It is reporting a different checksum than the one you provided, so the downloaded file must have been corrupted. I'll try to download a new copy. Thanks for your help. Barry Kumnick ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-30 14:54 Message: Logged In: YES user_id=31435 It sounds like you were trying to run md5sum with its "-c" switch: don't. Do it like so: C:\Code>\updates\md5sum Python-2.1.1.exe 39ef54d3e29ea402c8b693b4f3fedd4a *Python-2.1.1.exe C:\Code> There are no corrupt files in the installers we ship, so stop trying to pursue that path: you'll just get in worse trouble if you do. Something else is hosed on your system. BTW, you can open the installer (as a file!) in WinZip, and use WinZip's Action -> Test to test for corruption. md5 is a more powerful whole-file check, but the installer saves a CRC code for each individual file and that's what the WinZip Test looks at. If either md5 or WinZip complains, your installer got corrupted during (or maybe even after) downloading. ---------------------------------------------------------------------- Comment By: Barry Kumnick (barryk) Date: 2001-11-30 14:34 Message: Logged In: YES user_id=389562 Hi, Thanks for getting back to me. First I have a correction to make: it was the tk83.dll not the tcl83.dll that was reported as corrupt. Actually, I am running NT 4 Enterprise Server SP6 on a dual processor PIII box. I tried to install on another box running NT 4 Enterprise Server SP5 with a single processor, and got the same error. However, on that machine I was able to keep pressing retry, and the progress bar kept advancing. In fact it went up a few percent every time I pressed the retry button on the error dialog. It continued to advance past 100%. I stopped trying after it got to around 113% and it said the system needed to be rebooted to complete the installation. I did that. Apparently, everything installed except the tk83.dll. I also tried the md5sum.exe application Martin suggested, but it reports: md5sum: Python-2.1.1.exe: no properly formated MD5 checksum lines found. Do you have another suggestion for computing the MD5 checksum? I don't have md5sum.py on my system. BTW: With the current 2.1.1 installation, I was able to get the Python command shell to execute, but when attempting to launch IDLE, it fails with an error message saying the tk83.dll can not be found. Is there somewhere I can download the tk83.dll separately as a workaround? Thanks, Barry Kumnick ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-30 09:43 Message: Logged In: YES user_id=31435 Thanks for finding that program, Martin! This should be received much better than my usual advice to compute MD5 with md5sum.py . ---------------------------------------------------------------------- Comment By: Martin v. Löwis (loewis) Date: 2001-11-30 08:19 Message: Logged In: YES user_id=21627 Can you please verify the md5sum of the file you've downloaded? It is 39ef54d3e29ea402c8b693b4f3fedd4a Python-2.1.1.exe You can get an md5sum executable from http://etree.org/md5com.html ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2001-11-29 18:25 Message: Logged In: YES user_id=31435 Tens of thousands of Windows users have run this installer without problems, so-- and especially given your other bug report about 2.2b2 --the presumption has to be that something is uniquely screwed up on the specific box you tried it on. Can you reproduce on another (physically distinct) machine? ---------------------------------------------------------------------- You can respond by visiting: http://sourceforge.net/tracker/?func=detail&atid=105470&aid=487317&group_id=5470